{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "90adfce1",
   "metadata": {
    "colab_type": "text",
    "id": "view-in-github"
   },
   "source": [
    "<a href=\"https://colab.research.google.com/github/tomasonjo/blogs/blob/master/lp-combiner/Link%20prediction%20combiner.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "R5pT-OxY1qCZ",
   "metadata": {
    "id": "R5pT-OxY1qCZ"
   },
   "source": [
    "* Updated to GDS 2.0 version\n",
    "* Link to original blog post: https://towardsdatascience.com/a-deep-dive-into-neo4j-link-prediction-pipeline-and-fastrp-embedding-algorithm-bf244aeed50d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "UOOmOa2513GX",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "UOOmOa2513GX",
    "outputId": "2042fec8-e7d7-4716-c097-446fa698eb16"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting neo4j\n",
      "  Downloading neo4j-4.4.2.tar.gz (89 kB)\n",
      "\u001b[?25l\r",
      "\u001b[K     |███▋                            | 10 kB 17.6 MB/s eta 0:00:01\r",
      "\u001b[K     |███████▎                        | 20 kB 6.0 MB/s eta 0:00:01\r",
      "\u001b[K     |███████████                     | 30 kB 8.0 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████▋                 | 40 kB 9.9 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████▎             | 51 kB 5.4 MB/s eta 0:00:01\r",
      "\u001b[K     |██████████████████████          | 61 kB 6.3 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████▋      | 71 kB 7.2 MB/s eta 0:00:01\r",
      "\u001b[K     |█████████████████████████████▎  | 81 kB 5.7 MB/s eta 0:00:01\r",
      "\u001b[K     |████████████████████████████████| 89 kB 3.9 MB/s \n",
      "\u001b[?25hRequirement already satisfied: pytz in /usr/local/lib/python3.7/dist-packages (from neo4j) (2018.9)\n",
      "Building wheels for collected packages: neo4j\n",
      "  Building wheel for neo4j (setup.py) ... \u001b[?25l\u001b[?25hdone\n",
      "  Created wheel for neo4j: filename=neo4j-4.4.2-py3-none-any.whl size=115365 sha256=b12b1a7e7947731aeaf41cb94259b9dc369d5064b5032312e908525ac59cf703\n",
      "  Stored in directory: /root/.cache/pip/wheels/10/d6/28/95029d7f69690dbc3b93e4933197357987de34fbd44b50a0e4\n",
      "Successfully built neo4j\n",
      "Installing collected packages: neo4j\n",
      "Successfully installed neo4j-4.4.2\n"
     ]
    }
   ],
   "source": [
    "!pip install neo4j"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8eb262b4",
   "metadata": {
    "id": "8eb262b4"
   },
   "outputs": [],
   "source": [
    "# Define Neo4j connections\n",
    "import pandas as pd\n",
    "from neo4j import GraphDatabase\n",
    "host = 'bolt://44.193.28.203:7687'\n",
    "user = 'neo4j'\n",
    "password = 'combatants-coordinates-tugs'\n",
    "driver = GraphDatabase.driver(host,auth=(user, password))\n",
    "\n",
    "def run_query(query, params={}):\n",
    "    with driver.session() as session:\n",
    "        result = session.run(query, params)\n",
    "        return pd.DataFrame([r.values() for r in result], columns=result.keys())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4824ecc6",
   "metadata": {
    "id": "4824ecc6"
   },
   "source": [
    "# A deep dive into Neo4j Link Prediction pipeline and FastRP embedding algorithm\n",
    "### Learn how to train and optimize Link Prediction models in the Neo4j Graph Data Science library to get the best results\n",
    "In my previous blog post, I introduced the newly available Link Prediction pipeline in the Neo4j Graph Data Science library. Since the post, I took more time to dig deeper and learn the inner workings of the pipeline. I've learned a couple of things along the way that I want to share with you. At first, I intended to show how the Link Prediction pipeline combines node properties to generate input features of the Link Prediction model. However, when I was developing the content, I noticed a couple of insights about using the FastRP embedding algorithm. Therefore, by the end of this blog post, you will hopefully learn more about the FastRP embedding model and how you can combine multiple node features as an input to the Link Prediction model.\n",
    "# Graph import\n",
    "I had to find a small network so I easily visualize results as we go along. I decided to use the interaction network from the first season of the Game of Thrones TV show made available by Andrew Beveridge.\n",
    "The graph model consists of characters and their interactions. We will treat the interaction relationship as undirected, where is character A interacts with character B, this directly implies that character B also interacted with character A. We also know how many times two characters interacted, and we store that information as the relationship property.\n",
    "If you want to follow along with examples in this post, I recommend using a Blank project in Neo4j Sandbox. It is a free cloud instance of Neo4j database that comes pre-installed with both APOC and Graph Data Science plugins.\n",
    "The dataset is available on GitHub, so we can easily import it into Neo4j with the following Cypher query:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "d115c636",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 49
    },
    "id": "d115c636",
    "outputId": "789632d3-9b8d-4d36-c473-367576b371b0"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: []\n",
       "Index: []"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "run_query(\"\"\"\n",
    "LOAD CSV WITH HEADERS FROM \"https://raw.githubusercontent.com/mathbeveridge/gameofthrones/master/data/got-s1-edges.csv\" as row\n",
    "MERGE (s:Character{name:row.Source})\n",
    "MERGE (t:Character{name:row.Target})\n",
    "MERGE (s)-[i:INTERACTS]-(t)\n",
    "SET i.weight = toInteger(row.Weight)\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c3b0550",
   "metadata": {
    "id": "0c3b0550"
   },
   "source": [
    "Link prediction pipeline\n",
    "Under the hood, the link prediction model in Neo4j uses a logistic regression classifier. We are dealing with a binary classification problem, where we want to predict if a link exists between a pair of nodes or not. On a high level, the link prediction pipeline follows the following steps:\n",
    "* Node Feature engineering\n",
    "* Link Feature combiner\n",
    "* Model training\n",
    "* Predict new links \n",
    "\n",
    "In this post, we will focus on the first two steps.\n",
    "As a first step, you have to define node features. For example, you could use custom node properties such as age or gender. You can also use graph algorithms such as PageRank or Betweenness centrality as initial node features. In this blog post, we will start by using FastRP node embeddings to define initial node features. The nice thing about the FastRP embedding algorithm is that it captures the network information and preserves the similarity in embedding space between neighboring nodes that are close in a graph. At the moment, you can't use pairwise information such as the number of common neighbors or the length of the shortest path between a pair of nodes as input features.\n",
    "In the second step, the link feature combiner creates a single feature from a pair of node properties. Currently, there are three techniques that you can use to combine a pair of node properties into a single link feature vector:\n",
    "* Cosine distance\n",
    "* L2 or Euclidian distance\n",
    "* Hadamard product\n",
    "\n",
    "All the available link feature combiner techniques are order-invariant as the Link Prediction pipeline supports predicting only undirected relationships at the moment. You can use multiple link feature combiners in a single pipeline to define several feature vectors, which are then concatenated as an input to the Link Prediction model. I'll walk you through an example later in the post.\n",
    "Once the node features and link feature combiner are defined, you can train the model to predict new connections.\n",
    "# Node Feature engineering\n",
    "You can preprocess node features before defining the Link Prediction pipeline. You can also include them directly in the pipeline definition if you only use graph algorithms such as node embeddings or centrality measures as node features. In the first example, we will use the FastRP embeddings as our Link Prediction model node features. Therefore, we could potentially include them in the pipeline definition. However, we will first do a short analysis of the node embedding results, so we need to store the node embeddings to the graph before diving into the pipeline definition. We start by projecting an undirected named graph. Take a look at the documentation for more information about the inner workings of the Graph Data Science library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "f9ee72e1",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81
    },
    "id": "f9ee72e1",
    "outputId": "1ba2ee13-34da-4f9f-ea9e-06eeb60f9f6f"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>nodeProjection</th>\n",
       "      <th>relationshipProjection</th>\n",
       "      <th>graphName</th>\n",
       "      <th>nodeCount</th>\n",
       "      <th>relationshipCount</th>\n",
       "      <th>projectMillis</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>{'Character': {'label': 'Character', 'properti...</td>\n",
       "      <td>{'INTERACTS': {'orientation': 'UNDIRECTED', 'i...</td>\n",
       "      <td>gots1</td>\n",
       "      <td>126</td>\n",
       "      <td>1098</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                      nodeProjection  \\\n",
       "0  {'Character': {'label': 'Character', 'properti...   \n",
       "\n",
       "                              relationshipProjection graphName  nodeCount  \\\n",
       "0  {'INTERACTS': {'orientation': 'UNDIRECTED', 'i...     gots1        126   \n",
       "\n",
       "   relationshipCount  projectMillis  \n",
       "0               1098             11  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "run_query(\"\"\"\n",
    "CALL gds.graph.project('gots1', 'Character', {INTERACTS:{orientation:'UNDIRECTED', properties:'weight'}})\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "13fa96bc",
   "metadata": {
    "id": "13fa96bc"
   },
   "source": [
    "We will use Louvain, a community detection algorithm, to help us better understand the results of the FastRP embedding algorithm. You can use the following Cypher query to store the community structure information back to the database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "28e06535",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 142
    },
    "id": "28e06535",
    "outputId": "1f937ace-c7a8-473c-9259-8a9fcb580a63"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>writeMillis</th>\n",
       "      <th>nodePropertiesWritten</th>\n",
       "      <th>modularity</th>\n",
       "      <th>modularities</th>\n",
       "      <th>ranLevels</th>\n",
       "      <th>communityCount</th>\n",
       "      <th>communityDistribution</th>\n",
       "      <th>postProcessingMillis</th>\n",
       "      <th>preProcessingMillis</th>\n",
       "      <th>computeMillis</th>\n",
       "      <th>configuration</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>114</td>\n",
       "      <td>126</td>\n",
       "      <td>0.52832</td>\n",
       "      <td>[0.4918027508950885, 0.5283200895726703]</td>\n",
       "      <td>2</td>\n",
       "      <td>7</td>\n",
       "      <td>{'p99': 47, 'min': 2, 'max': 47, 'mean': 18.0,...</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>844</td>\n",
       "      <td>{'maxIterations': 10, 'writeConcurrency': 4, '...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   writeMillis  nodePropertiesWritten  modularity  \\\n",
       "0          114                    126     0.52832   \n",
       "\n",
       "                               modularities  ranLevels  communityCount  \\\n",
       "0  [0.4918027508950885, 0.5283200895726703]          2               7   \n",
       "\n",
       "                               communityDistribution  postProcessingMillis  \\\n",
       "0  {'p99': 47, 'min': 2, 'max': 47, 'mean': 18.0,...                     3   \n",
       "\n",
       "   preProcessingMillis  computeMillis  \\\n",
       "0                    0            844   \n",
       "\n",
       "                                       configuration  \n",
       "0  {'maxIterations': 10, 'writeConcurrency': 4, '...  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "run_query(\"\"\"\n",
    "CALL gds.louvain.write('gots1', {writeProperty:'louvain', relationshipWeightProperty:'weight'})\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41a45d6c",
   "metadata": {
    "id": "41a45d6c"
   },
   "source": [
    "Throughout this blog post, I will be using Neo4j Bloom to visualize the results of algorithms and link predictions. Take a look at this guide if you want to learn how to visualize networks with Bloom.\n",
    "\n",
    "Now we can go ahead and execute the FastRP embedding algorithm. The algorithm will produce an embedding or a fixed-size vector for every node in the graph. My friend CJ Sullivan wrote an [excellent article](https://towardsdatascience.com/behind-the-scenes-on-the-fast-random-projection-algorithm-for-generating-graph-embeddings-efb1db0895) explaining the inner working of the FastRP algorithm."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "e3d2d3f9",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81
    },
    "id": "e3d2d3f9",
    "outputId": "634448a1-0fc1-4bb2-e18f-58731bcb2a22"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>nodeCount</th>\n",
       "      <th>nodePropertiesWritten</th>\n",
       "      <th>preProcessingMillis</th>\n",
       "      <th>computeMillis</th>\n",
       "      <th>writeMillis</th>\n",
       "      <th>configuration</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>126</td>\n",
       "      <td>126</td>\n",
       "      <td>0</td>\n",
       "      <td>14</td>\n",
       "      <td>24</td>\n",
       "      <td>{'writeConcurrency': 4, 'nodeSelfInfluence': 0...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   nodeCount  nodePropertiesWritten  preProcessingMillis  computeMillis  \\\n",
       "0        126                    126                    0             14   \n",
       "\n",
       "   writeMillis                                      configuration  \n",
       "0           24  {'writeConcurrency': 4, 'nodeSelfInfluence': 0...  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "run_query(\"\"\"\n",
    "CALL gds.fastRP.write('gots1', {writeProperty:'embedding', embeddingDimension:56, relationshipWeightProperty:'weight'})\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a37ee29e",
   "metadata": {
    "id": "a37ee29e"
   },
   "source": [
    "First, we will evaluate FastRP embeddings with a t-SNE scatter plot visualization. The stored node embeddings are vectors with a length of 56, as defined by the embeddingDimension parameter. The t-SNE algorithm is a dimensionality reduction algorithm, which we can use to reduce the embedding dimension to two. Having vectors with length two allows us to visualize them with a scatter plot. The Python code I used for dimensionality reduction and scatter plot visualization is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b3211215",
   "metadata": {
    "id": "b3211215"
   },
   "outputs": [],
   "source": [
    "from sklearn.manifold import TSNE\n",
    "from matplotlib import pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "\n",
    "def tsne(embeddings, hue=None):\n",
    "    tsne = TSNE(n_components=2, n_iter=300)\n",
    "    tsne_results = tsne.fit_transform(embeddings['embedding'].to_list())\n",
    "\n",
    "    embeddings['tsne_x'] = [x[0] for x in list(tsne_results)]\n",
    "    embeddings['tsne_y'] = [x[1] for x in list(tsne_results)]\n",
    "\n",
    "    plt.figure(figsize=(16,10))\n",
    "    sns.scatterplot(\n",
    "        x=\"tsne_x\", y=\"tsne_y\",\n",
    "        hue=hue,\n",
    "        palette=\"deep\",\n",
    "        data=embeddings,\n",
    "        legend=\"full\",\n",
    "        alpha=0.9\n",
    "    )\n",
    "    # Add captions\n",
    "    #for i in range(embeddings.shape[0]):\n",
    "    #    plt.text(x=embeddings['tsne_x'][i]+0.3,y=embeddings['tsne_y'][i]+0.3,s=embeddings.character[i], \n",
    "    #          fontdict=dict(color='black',size=10),)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "a792ad8b",
   "metadata": {
    "id": "a792ad8b"
   },
   "outputs": [],
   "source": [
    "tsne_input = run_query(\"\"\"\n",
    "MATCH (c:Character)\n",
    "RETURN c.name as character, c.embedding as embedding, c.louvain as hue \n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "84a90a1c",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 675
    },
    "id": "84a90a1c",
    "outputId": "6b0a0cd5-4bf1-4216-c378-5d7cfc2e2864"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/tomaz/.local/lib/python3.8/site-packages/sklearn/manifold/_t_sne.py:795: FutureWarning: The default initialization in TSNE will change from 'random' to 'pca' in 1.2.\n",
      "  warnings.warn(\n",
      "/home/tomaz/.local/lib/python3.8/site-packages/sklearn/manifold/_t_sne.py:805: FutureWarning: The default learning rate in TSNE will change from 200.0 to 'auto' in 1.2.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA7gAAAJNCAYAAAAbEdlFAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAB07ElEQVR4nOz9eXic1Z3nf39O7VVSad8ty/Ii7zYGmy2AMZidBEJIMiH7nnQnHTKTng7zpGd6Jt0zQ7qnO+HXIQs0SUgnTRISthAg7PtqAzYG77tkWfuuUq3n+UOybFmSLVsl3VLp/bouX6761r18C8qSPjrnPrex1goAAAAAgOnO5XQDAAAAAACkAwEXAAAAAJARCLgAAAAAgIxAwAUAAAAAZAQCLgAAAAAgIxBwAQAAAAAZweN0AxOhqKjIVldXO90GAAAAAGACbNy4sdlaW3x8PSMDbnV1tTZs2OB0GwAAAACACWCM2T9SnSnKAAAAAICMQMAFAAAAAGQEAi4AAAAAICNk5DW4I4nH46qtrVVfX5/TraRVIBBQZWWlvF6v060AAAAAgKNmTMCtra1VOBxWdXW1jDFOt5MW1lq1tLSotrZWc+fOdbodAAAAAHDUjJmi3NfXp8LCwowJt5JkjFFhYWHGjUoDAAAAwOmYMQFXUkaF2yMy8T0BAAAAwOmYUQE3Xfbt26fly5c73QYAAAAA4BgEXAAAAABARiDgnqZkMqkvfelLWrZsma644gpFIhGtW7dOGzZskCQ1Nzerurp6cNv/+l//q84++2ytXLlSP/3pTx3sHAAAAAAyEwH3NO3cuVNf+9rX9O677yovL09/+MMfRt32rrvuUm5urt544w298cYbuvPOO7V3795J7BYAAAAAMt+MuU1Qus2dO1erVq2SJK1evVr79u0bddvHH39cmzdv1u9//3tJUkdHh3bu3MmtfQAAAAAgjQi4p8nv9w8+drvdikQi8ng8SqVSkjTk1j3WWv3rv/6rrrzyyknvEwAAAABmCqYop1F1dbU2btwoSYOjtZJ05ZVX6sc//rHi8bgkaceOHerp6XGkRwAAAADIVATcNPrrv/5r/fjHP9aZZ56p5ubmwfoXv/hFLV26VGeddZaWL1+ur3zlK0okEg52CgAAAACZx1hrne4h7dasWWOPrGZ8xNatW7VkyRKHOppYmfzeAAAAAOB4xpiN1to1x9cnZQTXGPMzY0yjMWbLMbUCY8wTxpidA3/nj7LvZwa22WmM+cxk9AsAAAAAmH4ma4ryLyRddVztFklPWWtrJD018HwIY0yBpL+TdK6kcyT93WhBGAAAAAAws01KwLXWPi+p9bjy9ZLuHnh8t6QPjrDrlZKesNa2WmvbJD2h4UEZAAAAAABHbxNUaq2tH3h8WFLpCNvMknTwmOe1AzUAAAAAGBObTCjWckipvi55corlzStxuiVMkClxH1xrrTXGjGu1K2PMlyV9WZKqqqrS0hcAAACA6cFaq3hbvZJdbXJn58lbUC5jXErFY+ra9JTaX3lASiVlfEEVX/MVBecsd7plTAAnbxPUYIwpl6SBvxtH2KZO0uxjnlcO1Iax1t5hrV1jrV1TXFyc9mYBAAAATE3WWvXueF2H7/kHNT7wfdXf8/fq2fqKbCqpeNMBtb/0BymV7N82FlHLE79Qorvd2aYxIZwMuA9JOrIq8mckPTjCNn+WdIUxJn9gcakrBmrTVjKZ1Jlnnqn3v//9kqSLLrpIq1at0qpVq1RRUaEPfvCDzjYIAAAATDPxtsNqefJu2USsv5BMqOXpf1e8tV7xrpZh2yd72pXs7ZjkLjEZJmWKsjHmHknrJBUZY2rVvzLyrZJ+Z4z5gqT9kj46sO0aSV+11n7RWttqjPl7SW8MHOq71trjF6uaVm677TYtWbJEnZ2dkqQXXnhh8LUbb7xR119/vVOtAQAAANNSsrv9aLgdLCaU7G6XJ1w4bHt3Vp7codxJ6g6TaVICrrX2plFeWj/CthskffGY5z+T9LMJam1Uz248qF8+ulXNbREV5Qf16auXaN3q2Sff8QRqa2v1pz/9Sd/5znf0L//yL0Ne6+zs1NNPP62f//zn4zoHAAAAMNO4s/NkPL6hIdftkTs7T57cEuW970a1v/rA4DW4hZd9Vp7sPKfaxQSaEotMTTXPbjyoH967SdF4/zz9praIfnjvJkkaV8j95je/qX/8x39UV1fXsNceeOABrV+/Xjk5Oad9fAAAAGAm8uaXqXD9Z9Ty1MA0ZbdHhZd+qn+hKZdbOWddrkD1MqV6u+TJLZI3b6QbuCATEHBH8MtHtw6G2yOi8aR++ejW0w64Dz/8sEpKSrR69Wo9++yzw16/55579MUvfnH4jgAAAABOyBij0KJz5C2erWR3m9xZefIW9q+iLEnG7ZG/mDutzAQE3BE0t0VOqT4WL730kh566CE98sgj6uvrU2dnpz75yU/qV7/6lZqbm/X666/r/vvvP+3jAwAAADOZMUa+wgqpsMLpVuAgJ1dRnrKK8oOnVB+L//t//69qa2u1b98+/eY3v9Gll16qX/3qV5Kk3//+93r/+9+vQCBw2scHAAAAgJmOgDuCT1+9RH6ve0jN73Xr01cvmZDz/eY3v9FNN422DhcAAAAAYCyYojyCI9fZpnsV5cHjr1undevWDT4f6ZpcAAAA4GQ6OyJqauiWyxgVl2UrO8yMQMxsBNxRrFs9O22BFgAAAEi3psNdevA3b6uzo0+SVFSSrfd/ZKUKirIGt+lsj6ip8UgADis77HeqXWBSEHABAACAaWjLW3WD4VaSmhu7tXt7owqK5kqSGgcCcNfANiVlYV174wrlHxOAgUzDNbgAAADANJNMpFR3oH1Yvb62U5JkrdU7G2sHw63UH3j37GiarBanFZtMKN7eqERX69i2t6kJ7ginixFcAAAAYJpxe1xasLhEjYe7htTnLiiUNHoAPnyoczLam1biHc3qfOMRdW99SS6vX7nnf1DZS94nl2/o9cw2lVRf7XZ1bX5Gtq9H2SsvUbB6xbDt4CxGcAEAAIBpaPGKMs1bWCRJMkZaekaFqmv6n3u8bi1YXDJsn6p5hZPa43TQ/e4L6n73eSmVVCraq7Zn/0N9dTuGbRc9tEuND/xAkd1vqa9uh5of/akiezY50DFOhBFcAAAAYBrKKwjpmg+tUFtLr4yRCoqz5PEcvdXlkjPK1VjfqT07m2WMtOyMClUvIOAeKxnpUs+2V4bVo3U7FJq7ckgtsu8d6bipyR1vPa7g/FVyeVm8a6og4E6y2267TXfeeaestfrSl76kb37zm3r77bf11a9+VX19ffJ4PPrRj36kc845x+lWAQAAMMX5/B6VVuSM+Fp+QUjX3DgQgF1GBYUhebzuEbedqYzXL09uiZLHXXvrDo/wiwAzfPKrcbn7h88xZTBFeRJt2bJFd955p15//XVt2rRJDz/8sHbt2qW/+Zu/0d/93d/p7bff1ne/+139zd/8jdOtAgAAIAMcCcAlZWHC7QhcHp/yzrlWch8d93PnlihYtWTYtsG5KyTX0P+GOWddIZfHN+F9YuwYwR1F15bn1fbMr5XobJEnp1D5l3xC4eVrx3XMrVu36txzz1UoFJIkXXzxxbrvvvtkjFFnZ/8F/x0dHaqoqBh3/wAAAABOLjB7ico++t8Uazool8cnX+lcefOKh23nL1+g0hu+pZ7trykV7VXW4vMUmD08CMNZBNwRdG15Xs1/+olsIipJSnQ2q/lPP5GkcYXc5cuX6zvf+Y5aWloUDAb1yCOPaM2aNfrBD36gK6+8Un/913+tVCqll19+OS3vAwAAAMDJ+UvmyF8y54TbGGMUqFyoQOXCSeoKp4MpyiNoe+bXg+H2CJuIqu2ZX4/ruEuWLNG3v/1tXXHFFbrqqqu0atUqud1u/fjHP9b3v/99HTx4UN///vf1hS98YVznAQAAADA5oof3qv21h9T24u8VObhNNplwuqUZjYA7gkRnyynVT8UXvvAFbdy4Uc8//7zy8/O1cOFC3X333frQhz4kSfrIRz6i119/fdznAQAAADCxoof3qOEP/6SOVx9S58bH1Hj/P6vvwHtOtzWjEXBH4MkZefn00eqnorGxUZJ04MAB3Xffffr4xz+uiooKPffcc5Kkp59+WjU1NeM+DwAAAICJ1btnk2widrRgrTo2/lk2wSiuU7gGdwT5l3xiyDW4kmQ8fuVf8olxH/vGG29US0uLvF6vbr/9duXl5enOO+/UzTffrEQioUAgoDvuuGPc5wEAAAAwcWwyIXd2nrJXXCzjcqtn15tK9bQrFY3I2qQMUcsR/FcfwZGFpNK9irIkvfDCC8NqF154oTZu3DjuYwMAAACYeDaZUNc7z6n1mV8rFemUXB7lnf9B9Wx/TTmrLpXL63e6xRmLgDuK8PK1aQm0AAAAADJLrKVObc//VsblksufJRuPqmPDIyq57htT6tZBNpVUrKWuf9AuK0/eokq5PF6n25pQBFwAAAAAOAXJ7nbJpiRJxuOVcffHKncwW+5AloOdDdWz/XW1PPkLKZWUZJT3vhsUPvPyjA65LDIFAAAAYEaxqaSi9XvUteUF9e5+S4me9lPa35NTKLmPGSs0RsYfkju7IL2NjkO8rUGtz/xqINxKklX7y/cr3nzQ0b4mGiO4AAAAAGaU3l1vqvmxOyRrJUmBqiUqvPzz8mTnj2l/b0G5Ci/7rFqf/nfZeFQuf0iFl39O3rySiWz7lCQjnbLx6HFVq2RPhyP9TBYCLgAAAIAZI9HTrrbnfzMYbiWp78BWxQ7vlWfB2AKucbmVvfg8+UurlezpkDtcIG9u8US1fFo82QVyBbOVinQfLbrccucUOdfUJGCKMgAAAIAZw8b6lOzpHFZP9vWc8rG8+WUKVC6acuFW6p9GXXTlF+UK5UiSjC+gwss/J19hhcOdTSwC7iS77bbbtHz5ci1btkw/+MEPJEmbNm3S+eefrxUrVugDH/iAOjuH/4MDAAAAMH7u7HwFq5cPLRqTkcEvOGe5yj/2HZV+5BaV3/Tflb34PBmX2+m2JhQBdxJt2bJFd955p15//XVt2rRJDz/8sHbt2qUvfvGLuvXWW/XOO+/ohhtu0D/90z853SoAAACQkVxev/Iu/LCCc1ZIMnKHclV05RflK5njdGsTwhMuVKBigbx5pU63Mim4BncUL+x/XfdsflAtva0qDBXoppXX66I554zrmFu3btW5556rUCgkSbr44ot13333aceOHVq7tv+eu5dffrmuvPJK/f3f//243wMAAACA4XyFs1R07VeV6G6VyxuUJzvP6ZaQJozgjuCF/a/rp2/8Ws29rbKSmntb9dM3fq0X9r8+ruMuX75cL7zwglpaWtTb26tHHnlEBw8e1LJly/Tggw9Kku69914dPJjZS3cDAAAATnN5/fLllxNuMwwBdwT3bH5QsWRsSC2WjOmezQ+O67hLlizRt7/9bV1xxRW66qqrtGrVKrndbv3sZz/Tj370I61evVpdXV3y+XzjOg8AAAAAzEQE3BG09LaeUv1UfOELX9DGjRv1/PPPKz8/XwsXLtTixYv1+OOPa+PGjbrppps0f/78cZ8HAAAAAGYaAu4ICkMFp1Q/FY2NjZKkAwcO6L777tPHP/7xwVoqldI//MM/6Ktf/eq4zwMAAADg9MQ7m9V3aKfi7Y1Ot4JTxCJTI7hp5fX66Ru/HjJN2ef26aaV14/72DfeeKNaWlrk9Xp1++23Ky8vT7fddptuv/12SdKHPvQhfe5znxv3eQAAAACcut69m9Ty+M+V6uvuv3fspZ9WaOEaGcPY4HRAwB3BkdWS072KsiS98MILw2o333yzbr755nEfGwAAAICUivcpdniv4m2H5c7Kk798vtyhnJPuF+9oUsuff6ZUtEeSZGN9an7iZyovmiVf4ayJbhtpQMAdxUVzzklLoAUAAAAweay16t7yotqe/81gLbhgtQov/ZTcwewT7pvsah0Mt0eLCSU6mgm40wTj7AAAAAAyRqK9Qe0v3zekFtm1UbGmAyfd1xXKkfEcd0cT45KbWwlNGwRcAAAAABkjFeuTTcSG16O9J93Xm1+mgnUfl1zu/oIxyr/ww/IVVqS7TUwQpigDAAAAyBie3GL5imcr1nRwsGbcXnnzy066rzFGWUvOl7e4SsmuFrmz8+QrrJRxeyeyZaQRI7gAAAAAMoY7kKWCyz6nQNVSSUae/DIVf+Br8hVVjml/43LLX1Kl0Pwz5S+dK+Mh3E4njOACAAAAyCj+kioVv/9rSvZ0yOUPnXRxKUlKdLcr1rhfNhGTt2iWfAVMS56OGMGdRJ///OdVUlKi5cuXD9ZaW1t1+eWXq6amRpdffrna2tok9a/+9o1vfEMLFizQypUr9eabbzrVNgAAADDtuLx+efNKxhRu4+1Nanr4h2r647+q+dGf6vBv/4/66namrZdkpFt9h3Yp2rhfqRGuD0b6EHAn0Wc/+1k99thjQ2q33nqr1q9fr507d2r9+vW69dZbJUmPPvqodu7cqZ07d+qOO+7QX/zFXzjRMgAAAJDx+mq3Kdawb/C5jfWp842HlUrEZK1VvKNJ8fZG2VTylI8da65T4/3/rIZ7b9Xhe/5Bbc/9RsmejjR2j2MxRXkUjc89rwP//mtFm1vkLypU1ac+oZKL147rmGvXrtW+ffuG1B588EE9++yzkqTPfOYzWrdunb73ve/pwQcf1Kc//WkZY3Teeeepvb1d9fX1Ki8vH1cPAAAAAIZKdDQOq8Wa65ToblfvtlfVufExWZtSeOUlyjnrSnnGeNsgm0qq8+0njlnwyqp7y/MKVC5S1qJz0/cGMIgR3BE0Pve8dt/+E0WbmiVrFW1q1u7bf6LG555P+7kaGhoGQ2tZWZkaGhokSXV1dZo9e/bgdpWVlaqrq0v7+QEAAICZzl82d1gtVLNasfrd6njtof7bDiUT6nrrCfXu2jDm46aiverb/+6werRh/7j6xegIuCM48O+/VioaHVJLRaM68O+/ntDzGmNkjJnQcwAAAAAYyj9rkXLPu37wdkDBuWcovOoyRfa8NWzbnq2vyCYTYzquyxeUv2LBsLqvaNb4GsaomKI8gmhzyynVx6O0tHRw6nF9fb1KSkokSbNmzdLBg0fv3VVbW6tZs/iHAAAAAKSbO5Cl3HOuVdbCs2UTCXnyiuXy+uUpqJA0dLFXb1Gl5HKP6bjG7VHOWVcpWrdLyZ7+xWSD1csVmL043W8BAxjBHYG/qPCU6uNx3XXX6e6775Yk3X333br++usH67/85S9lrdWrr76q3Nxcrr8FAABAxklGutWz4w21PvNrdb79lOJtDY70YYxL3vwy+Yor5fL6JUlZC1bLFco9uo0vqPDyi09p1qW/dI7K/tN/U8n1N6v0xr9W4ZVfkiec/lyBfozgjqDqU5/Q7tt/MmSassvvV9WnPjGu495000169tln1dzcrMrKSv2v//W/dMstt+ijH/2o7rrrLs2ZM0e/+93vJEnXXHONHnnkES1YsEChUEg///nPx3VuAAAAYKqxqaQ633pcnW88MljrLqpUyXU3yxPOd7Czfr7i2Sr78N8o1rBP1qbkK5kjX+Gp3x/XEy6QJ1wwAR3ieATcERxZLTndqyjfc889I9afeuqpYTVjjG6//fZxnQ8AAACYyuLtjep88/GhteZaxZr2T4mAK0ne/FJ580udbgNjRMAdRcnFa8cdaAEAAACcQCopJYffW9Ym4w40g0zANbgAAAAAHOHJK1Fw/plDai5/SL6i2aPsAZwYI7gAAAAAHOHy+JR/4YflzS9V784N8hVXKXzWFfLmlzndGqYpAi4AAAAAx3jzSpR/wY3KWX2VXF6/jJuIgtPHpwcAAACA49yBLKdbQAbgGlwAAAAAQEYg4E6iz3/+8yopKdHy5csHa62trbr88stVU1Ojyy+/XG1tbZKkjo4OfeADH9AZZ5yhZcuWcR9cAAAAADgJAu4k+uxnP6vHHntsSO3WW2/V+vXrtXPnTq1fv1633nqrJOn222/X0qVLtWnTJj377LP61re+pVgs5kTbAAAAADAtcA3uKN7ZWKunH92ujraIcvODuvTqRVqxunJcx1y7dq327ds3pPbggw/q2WeflSR95jOf0bp16/S9731Pxhh1dXXJWqvu7m4VFBTI4+F/FwAAAACMxtERXGPMImPM28f86TTGfPO4bdYZYzqO2eZ/THRf72ys1cP3vqOOtogkqaMtoofvfUfvbKxN+7kaGhpUXl4uSSorK1NDQ4Mk6etf/7q2bt2qiooKrVixQrfddptcLgbcAQAAAGA0jiYma+12a+0qa+0qSasl9Uq6f4RNXziynbX2uxPd19OPblc8nhxSi8eTevrR7RN6XmOMjDGSpD//+c9atWqVDh06pLfffltf//rX1dnZOaHnBwAAAIDpbCoNCa6XtNtau9/pRo6M3I61Ph6lpaWqr6+XJNXX16ukpESS9POf/1wf+tCHZIzRggULNHfuXG3bti3t5wcAAACATDGVAu7HJN0zymvnG2M2GWMeNcYsG2kDY8yXjTEbjDEbmpqaxtVIbn7wlOrjcd111+nuu++WJN199926/vrrJUlVVVV66qmnJPVPY96+fbvmzZuX9vMDAAAAQKaYEgHXGOOTdJ2ke0d4+U1Jc6y1Z0j6V0kPjHQMa+0d1to11to1xcXF4+rn0qsXyet1D6l5vW5devWicR33pptu0vnnn6/t27ersrJSd911l2655RY98cQTqqmp0ZNPPqlbbrlFkvTf//t/18svv6wVK1Zo/fr1+t73vqeioqJxnR8AAAAAMtlUWZb3aklvWmsbjn/BWtt5zONHjDE/MsYUWWubJ6qZI6slp3sV5XvuGXmA+shI7bEqKir0+OOPj+t8AAAAAJyT6OmQUkm5s/MH19rBxJoqAfcmjTI92RhTJqnBWmuNMeeof9S5ZaIbWrG6ctyBFgAAAMDMk4r1qWfHG+p49QGl4jGFV16i8BmXyJOd73RrGc/xgGuMyZJ0uaSvHFP7qiRZa38i6cOS/sIYk5AUkfQxa611olcAAAAAOJm+uu1qferuweedGx6Ryx9U7pqrHexqZnA84FpreyQVHlf7yTGPfyjph5PdFwAAAADnxJoPKlq/W0ql5CubK3/pXKdbGrO+A1uH1brffVHZyy+WOxByoKOZw/GACwAAAADHijbsU8N9/ywbG7hNp9uj0hv+iwKzFjrb2Bi5s/OG1TzhQhmPd/KbmWGmxCrKAAAAAHBE7643j4ZbSUom1PX2U7I25VxTpyA4Z4VcoZyjBZdbOWuukmsKBtxUvE99dTvVs+01Ret3yybiTrc0LozgAgAAAJhSEl2tw2udzVIqJbknZozOWqtUPCqX1ydjxncOX9Esld74XxWt3yMlY/KVVMtXWp2eRtMolYirc+Pj6njtocFa/sU3KbxynYzLfYI9py5GcCfR5z//eZWUlGj58uWDtdbWVl1++eWqqanR5Zdfrra2NklSW1ubbrjhBq1cuVLnnHOOtmzZ4lTbAAAAwKTKWnDmsFr28rUy7okZn4u3HFLb87/V4Xv+Xq3P/FqxpoPjPqavoFzhZRcovPIS+cvmTsnbBMVbatXx2h+H1NpevFfxtsMOdTR+BNxJ9NnPflaPPfbYkNqtt96q9evXa+fOnVq/fr1uvfVWSdL/+T//R6tWrdLmzZv1y1/+UjfffLMTLQMAAACTzj97qQou+aTc2flyhXKU974bFZo/PPSmQzLSrebH71LX208q0d6g7neeU9OffqxET/uEnG8qSfZ0SjruBjXJhFKRbkf6SQemKI9i+1uv6NXH/6Cu9haF8wp13hU3atGZ54/rmGvXrtW+ffuG1B588EE9++yzkqTPfOYzWrdunb73ve/pvffe0y233CJJWrx4sfbt26eGhgaVlpaOqwcAAABgqnP7gwqvXKfggjMla+XJypuwc8Xb6hVr3D+kluhoVLzl0ISedyrw5BXLeHyyidhgzRUMy5NT5GBX48MI7gi2v/WKnrn/F+pqb5EkdbW36Jn7f6Htb72S9nM1NDSovLxcklRWVqaGhgZJ0hlnnKH77rtPkvT6669r//79qq2tTfv5AQAAgKnKE8qd8JBpXCOP+U3Xa1BPhTe/XEVXf3lwQSx3uFBFV39ZnpzCk+w5dTGCO4JXH/+DEvHYkFoiHtOrj/9h3KO4J2KMGZybf8stt+jmm2/WqlWrtGLFCp155plyuzP/HxkAAAAwmbwF5QrVnK3enW8M1gKzl8pbOMvBriaHMUaheavkK65SMtIld1aePFm5Trc1LgTcERwZuR1rfTxKS0tVX1+v8vJy1dfXq6SkRJKUk5Ojn//855L6V3SbO3eu5s2bl/bzAwAAADOZyxdQ/kUfUWDOMkUP7ZS/bJ6Cc5bJHcx2urVJ4wkXyBMucLqNtGCK8gjCeSMPyY9WH4/rrrtOd999tyTp7rvv1vXXXy9Jam9vVyzWP4r8b//2b1q7dq1ycnJGPQ4AAACA0+MJFyi87EIVXf45hVdcPK2vQZ3pCLgjOO+KG+Xx+obUPF6fzrvixnEd96abbtL555+v7du3q7KyUnfddZduueUWPfHEE6qpqdGTTz45uLDU1q1btXz5ci1atEiPPvqobrvttnGdGwAAAMDIbDKuaP1udb/7knr3blYy0uV0SzhNTFEewZHrbNO9ivI999wzYv2pp54aVjv//PO1Y8eOcZ0PAAAAwMn1bH9dLU/8QkdumRNaeI4K1n18Rk1TzhQE3FEsOvP8CV1QCgAAAIDz4h1Nanv+tzr2frC9O15X9tILFZyz1LnGcFoIuAAAAABmrFQ0olS0d1g92Te505RjTQcVHbgfr79srnwzYBXniUDABQAAADBjeXIK5SuerVjTwaNFl1u+/LJJ6yFav0cND/yLbKyv//T+kEpu+C/yl1ZPWg+ZgkWmAAAAAMxY7kCWCi77rHxl8/ufZ+Wq6OqvyFtUOaHnTXS3K9ZySMlYn7rffWEw3EpSKtqrnu2vTej5MxUjuAAAAABmNH/JHJV88JtKdrfK5c+SJztvws5lU0n17n5Lrc/+h1K9ncpaeqHirYeGbRdvb5iwHjIZI7gAAAAAZjy3Pyhf4awJDbdS/7W2zY/eoVRvpySpd/tr8lcskKwdsl3WwnMmtI9MRcCdRJ///OdVUlKi5cuXD9buvfdeLVu2TC6XSxs2bBisP/HEE1q9erVWrFih1atX6+mnn3aiZQAAAABpFG+rl2xq8LlNxhVvOaScc94v4wvK5Q8p74IbFZyz/ARHwWiYojyJPvvZz+rrX/+6Pv3pTw/Wli9frvvuu09f+cpXhmxbVFSkP/7xj6qoqNCWLVt05ZVXqq6ubrJbBgAAAJBG7kB4WC3WsFeF6z+t8IqLJUmecMFkt5UxCLij6HyvUS0v7leiMypPjl+FF85RztKScR1z7dq12rdv35DakiVLRtz2zDPPHHy8bNkyRSIRRaNR+f3+cfUAAAAAwDm+0jkK1axW786NAxWj/LUfI9SmCQF3BJ3vNarx8V2yif6pA4nOqBof3yVJ4w65p+MPf/iDzjrrLMItAAAAMM25g2EVrPukspZcoFRvl7wFZfKVzHG6rYxBwB1By4v7B8PtETaRUsuL+yc94L777rv69re/rccff3xSzwsAAABgYrhDYYXmrnS6jYxEwB1BojN6SvWJUltbqxtuuEG//OUvNX/+/Ek9NwAAAICRxVrqFGuqlcvjla+0munFUwgBdwSeHP+IYdaTM3lThNvb23Xttdfq1ltv1QUXXDBp5wUAAACmi2SkW6loj9yhHLl8wUk5Z1/dDjU+eJtsvD8veAtnqfjav5Q3vzQtx493NiveXCfj8cpXVCl3KCctx50puE3QCAovnCPjGfqfxnhcKrxwfHPjb7rpJp1//vnavn27Kisrddddd+n+++9XZWWlXnnlFV177bW68sorJUk//OEPtWvXLn33u9/VqlWrtGrVKjU2No7r/AAAAECmiBzcpsN/+Ccduvs7anzwNkUb9k74OVOJmDrf+NNguJWkeEud+mq3peX40YZ9avjtrWr647+q8f5/UfOjP1W8ozktx54pGMEdwZHrbNO9ivI999wzYv2GG24YVvvbv/1b/e3f/u24zgcAAABkonjbYTU9/EPZWJ8kKXpol5ofvUOlH71FnlDuhJ3XxqKKtRwaVk90NI3/2MmEOt98Qsne9sFaX+129R3cKm/uRWM+TjLSLZuIyZ2dJ2Nm3ngmAXcUOUtLHFkxGQAAAMCJxVsPD4bbIxIdTUq0N01owHUFsxVaeLa63hy6AKy/fPzr5aRifYod3j2sHm8+OKb9bTKhyL531PbSH5Ts6VD20gsUXrVe3tzicfc2ncy8SA8AAABgWnMFQiMU3RN+Ha4xRuEV6xSqWSMZI+P1K+99H5K/ctG4j+0KhBQcYWVlf9m8Me0fPbxXTX/6kRJth2VjEXW9/aS63nxC1qZOvnMGYQQXAAAAwLTiLZqtrKUXque9FwdreedeJ29B2cSfO69EhVd8QbnnXifj9siTWyxjzJj2jbc3Kt5cJ7nd8hVXyZOdN/iaMS5lr1ynWNMBRQ/tkoxR9vK1CsxeMqZjx5oOSNYOqXVvfUk5a66UJ1w45vc33c2ogGutHfOHb7qwx32IAQAAgEzn9geVf+GHFVpwlpJdrfLklchfNk/G5Z6U87s8XvkKK05pn+jhvWp86P9TKtIlSfIVV6nomq/Km3f0skhfQYWKP/ANJdoPSy6PfAXlMh7v2HryDx+9dgdzZDyTdyeYqWDGBNxAIKCWlhYVFhZmTMi11qqlpUWBQMDpVgAAAIBJ5Q5mKzTClN6pyKaS6tr05GC4lfpHXCP73pF31foh27oDIbnHOC35WP7yBfLklSrR3jBQMcq74ENyB7PH0/q0M2MCbmVlpWpra9XUNP4VzqaSQCCgyspKp9sAAAAAMIpUPKro4X3D6rHmurSdw5tXopLrbla0fqdSfb3ylVSlZfGr6WbGBFyv16u5c+c63QYAAACAGcbtDyk0/yx1bnx0SD1YuTCt5/Hml8ibP7PvBDNjAi4AAAAAnAqbSireWq9kd7vc2XnyFpSP6TrfVCKmRHuj5HLLm1ci43Ire/mFirfWKbJ3s+RyK7zyEgWqlo56jFhrvSL7NiveVKdA1RIF5yyTO5STzrc3TKKrVbGGfUrF++QrnCVvcdW0u7yTgAsAAAAAx7E2pZ5tr6nl6V9KyYTk9qjw0k8pa8n5Mmb0u63G2xvV9tLvFdn1luRyKefMy5Vz1pXy5pWq6KqvKN7eIONyy5tfKuMeOY4lulrV/PCPFG+rlyT1bHtZ4TOvUP6FN07YQlrxjmY1P/oTxRr29RfcHpV84K8UnLNsQs43UbgPLgAAAAAcJ95SfzTcSlIyodanf6V4a/0J9+t+7yVFdr0pyUqppDo3Pqa+g1slSS6fX/6SKvmKZo0abqX+BaiOhNsjujY9pXhbwyh7jF/00K6j4VaSkgm1v3yfktHeCTvnRCDgAgAAAMBxkj3tR8PtAJuMK9ndPvo+0V717twwrN5Xu+3UTp5KjlBLSTZ1asc5lVP2dgyrxdsbZWN9E3bOiUDABQAAADAmqURM0cb9ih7eq2Qs4nQ7E8qdnS/j8Q2pGY9P7uz8UfdxeXzylVQNq3sLZ53Sub1FlXIdd3ufUM1qefImbgEpb/HsYbVQzWq5s3In7JwTgYALAAAA4KQS3a1qfepXOnzPP+jwb/+3mv/0Y8XbJ27KrNO8BeUqXP+ZwZBrPD4Vrv+MvAXlo+5j3B7lnHm5XP6sY45ToeCc5ad27rxSlVz/TWUtfp+8BRXKPfc65b3vRrmOC9zp5C+fr4JLPtnfuzEKLVit3NVXTdg1vxPFWGud7iHt1qxZYzdsGD41AAAAAMDp6dryglqfuntILeecDyj//Osd6mjiWWsVbzt8dBXl/LIxrSocbzusWHOtjMsjX0mVPOGC0zx/SjYel8vnP639T0eis0WpREyenCK5PN5JO++pMsZstNauOb7OKsoAAAAATqrvwHvDapE9byn37KsndGTRScYY+QrKpVFGbZN9PYo17leyp0PevFL5Sqpk3B5588vkzS9Lw/ldMpMYbiXJk1M4qedLNwIuAAAAgJPylc1V7843htQCsxbKuKfuKN9ESvb1qu2Fe9Xz3osDFaPCyz+n7KXvc7SvmY5rcAEAAACcVGjeGfKVzBl87s4pUvayi8Y0ZTcTxZr2HxNuJcmq9bl7FG9vcqwnMIILAAAAYAy8eaUqvv4bijfVyqaS8hVVnva1pemWSsQl2UmdKp3q7RpWs7GIUrHpdd/YTEPABQAAADAmnlCuPHOmzm1jUvGYIge2qGvjn2WTSeWcdYWCc1fK5QtM+Lk9+aWSyz3knrWewgp5wtP7GtbpjinKAAAAAKalvtptan74R4rW71ascZ+aH7tDkf1bJuXcvqJKFV31JblCOUefX/Y5uY+7f60TMvFOOWPFCC4AAACAaaln26vDat2bnlZowVkyZmLH8ozLrayaNfKXzVOqr0fucIHcgayT7zhBbCqpvtrt6t78jFKxPmWvWKfgnGWTMpo9lRBwAQAAAExLI4U34w9JGnnhq2Rfr2wsIndWrow7PVHIEy6QpsC1yH11O9X4wA8km+p/fnCriq7+irIWnu1sY5OMgAsAAABgWspadK6633vp6HWwxqXwyktGXNk5sv89tb94r+LtDQrOW6Xcc94vX2HFJHc8cSJ7Ng2G2yM633pCwXlnZOx9ikdCwAUAAAAwLflnLVTph/5avXvellJJBeetUmBWzbDtok0H1fjH/09KJiRJvTteVyrSreL3f00un3+Su54gruGhvj/oz6zbOBFwAQAAAExLxhgFZtWMGGqPFW89NBhuj+g7+J4SnU3yFVVOZIunxCbj6qvbqcjed+TyBRScu1L+srlj2jc0b5W63n5qyKrO4bOukMvjnah2pyQCLgAAAICM5vKOcK2uLyDjnVqjt5F9W9T08I8k9a+C3PnW4yr90F+PKeT6Kxao5Ib/op5tr8hGI8pa8j4FZi+Z4I6nHgIuAAAAgIzmK61WoHKR+mq3D9byzr9B3txiB7saKhWPqXPDozoSbiXJxqOK7N8ypoBrjEvBykUKVi6awC6nPgIuAAAAgIzmycpV4RWfV7R+t5I9HfIWVMhfMd/ptoayKaVikeHlWJ8DzUxfBFwAAAAAGc8TLpQnXOh0G6Ny+QIKr7pMrU//+zFVo0D1csd6mo4IuAAAAAAwBYQWrJaMUdfbT8nlDypnzdUKVJx4Aa1TlYrHZNxuGZdbqURMye52ufxBuYPhtJ7HKQRcAAAAAJgC3MFshZevVdbCcySXK633r010t6pn+xvq2faKvPnlCi9fq85NTyqyd7M8ucXKX/sxBauXyxhX2s7phOndPQAAAABkGJcvkNZwa1NJdb75hNpfvFfx5lolezvV8uyv1LvrTclaJdob1fTw7Yo1HUzbOZ3i+AiuMWafpC5JSUkJa+2a4143km6TdI2kXkmftda+Odl9AgAAAMCxEl2tihx4V9FDu+Qvm6fgnOXy5Ey963wTna3q2vzM4HNf4Sy173hNLn+WrPrvJ6yUFG85JH/JHOcaTQPHA+6AS6y1zaO8drWkmoE/50r68cDfAAAAAOCIVKxPbS/8Tr07N0iSet57SYE5y1V01ZfkDmQ53N1xXEbG7ZFNJiRJqWRc7uwCJXs7pFRSMi65/EG5/CGHGx2/6TBF+XpJv7T9XpWUZ4wpd7opAAAAADNXrKVuMNwe0bd/i+ItdQ51NDpvTpFy11wz+NymUgqfcUl/uO0vyFtUJXdWrkMdps9UGMG1kh43xlhJP7XW3nHc67MkHTsZvHagVj9J/QEAAADAUEfC4XGOjJJONdnL18odLlBk/xYFKhaoY+Njyrvwo7LxiIzbq0R3mxKdLfKXVjvd6rhMhYB7obW2zhhTIukJY8w2a+3zp3oQY8yXJX1ZkqqqqtLdIwAAAJBRkpEuRQ/tVqKrRZ7cYvnL5skdzHa6rWnDW1AuX/HsIQszefLL5S2scLCrkcXbGtW15Vn17tqoQOVieXJLlOpuU9dbjw/ZLrzqMoc6TB/HA661tm7g70ZjzP2SzpF0bMCtkzT7mOeVA7Xjj3OHpDskac2aNXbCGgYAAACmuVSsT+0v36/uLUd/7A6fdYXyz79BxuN1sLPpwx0Mq/DKL6p7y/OK7NuiYNVSZa+4WJ6sPKdbGyKViKn9lT+od+dGSf3XCvcd2qnc869X+wv3Dm6XtfQC+Ypnj3aYacPRgGuMyZLkstZ2DTy+QtJ3j9vsIUlfN8b8Rv2LS3VYa5meDAAAAJymWEvdkHArSV1vPaGsRedO+1V0J5OvcJby135MuedH5fL6puQ9ZBPtTerdOfQmNMn2RnlyS1T64b9Ror1R7qxc+UrnyZ0Bi0w5PYJbKun+/jsBySPpP6y1jxljvipJ1tqfSHpE/bcI2qX+2wR9zqFeAQAAgIxgo5ERinbkOk7IGCO3L+B0G6MyHo+MxyubiA2tSwrMWijNWuhMYxPE0YBrrd0j6YwR6j855rGV9LXJ7AsAAADIZJ6CMrlCOUr1dg7W3OFCefLLHOwKE8GTW6Lcs69R+ysPDNa8RZXyZehIvdMjuAAAAAAmmTenSMXX/qXaX7xX0YZ98lfUKP+CG+XJznO6NaSZMUbZKy+Vt3CW+g7tlDevVIHZS+QJFzjd2oQw/QOkmWXNmjV2w4YNJ98QAAAAmMFSsYiSkR65g9lyTeFptsDxjDEbrbVrjq8zggsAAADMUC5fUC5f0Ok2gLSZest8AQAAAABwGhjBBQAAAIBxSEUjijXXysYi8hSUy5tb7HRLMxYBFwAAAABOU7K3U20v/l49W1+WJLkC2Sr+wNcUqKhxuLOZiSnKAAAAAHCaovW7B8OtJKX6utX2wu+Vio39nsLx9gZ1b31ZXZufUbR+t2wqORGtzgiM4AIAAADAaUp0tQ6rxZr2KxnpGdMCXvG2w2p44AdKdjb3F1xulXzg6wpWr0h3qzMCI7gAAAAAcJo8I1xvG5i1UO5QeEz799XuOBpuJSmVVPsrDygZ60tXizMKARcAAAAATpO/fL5y1lwtmf5o5cktVt77PiSX1z+m/ZO9ncNqie422Xg0rX3OFExRBgAAAIDT5A5kKe+86xWqOVs2HpUnr0SerNwx7+8vnzeslr30wlM6Bo5iBBcAAAAAxsG4PfKXVCkwq+aUg6m/okaFV35R7nCBjNev8Kr1Cq+4eII6zXyM4AIAAACAQ1wer7IXn6dg1VLZRFzucL6MYRzydBFwAQAAAMBh7lCO0y1kBH41AAAAAADICARcAAAAAEBGIOACAAAAADICARcAAAAAkBEIuAAAAACAjEDABQAAAABkBAIuAAAAACAjEHABAAAAABmBgAsAAAAAyAgEXAAAAABARiDgAgAAAAAyAgEXAAAAAJARCLgAAAAAgIzgcboBAAAAAMgk8Y4mxZsOSMYlX/EceXIKnG5pxiDgAgAAAECaRBv3q/HB/0+p3g5JkievVMUf+Jp8BRUOdzYzMEUZAAAAANKk590XBsOtJCXaG9S76y0HO5pZCLgAAAAAkAY2mVD08N5h9XjjvslvZoYi4AIAAABAGhi3R6GaNcPqgbkrHehmZiLgAgAAAECaZC08W1mLz5WMkVxuhVdeomD1CqfbmjFYZAoAAAAA0sSTU6SC9Z9VzuqrJWPkzSuVcRO7Jgv/pQEAAAAgjVwer3xFlU63MSMxRRkAAAAAkBEIuAAAAACAjEDABQAAAABkBK7BBQAAAIA0iLXUKdawX5LkK62Wr7DC4Y5mHgIuAAAAAIxTtGGfGu//F6WivZIklz9LJTf8Z/lLq51tbIZhijIAAAAAjFPP1pcHw60kpaI96tn2ioMdzUwEXAAAAAAYp1hL3bBafIQaJhYBFwAAAADGKWvx+cNqoUXnOdDJzEbABQAAAIBxCs1dqdxzr5Px+mW8fuWee51Cc1c63daMwyJTAAAAADBO7lCOcs/9gLKWXiBJ8uYUOtzRzETABQAAAIA0MMYQbB3GFGUAAAAAQEYg4AIAAAAAMgIBFwAAAACQEQi4AAAAAICMQMAFAAAAAGQEAi4AAAAAICMQcAEAAAAAGYGACwAAAADICARcAAAAAEBGIOACAAAAADICARcAAAAAkBEIuAAAAACAjEDABQAAAABkBAIuAAAAACAjOBpwjTGzjTHPGGPeM8a8a4y5eYRt1hljOowxbw/8+R9O9AoAAAAAmNo8Dp8/Ielb1to3jTFhSRuNMU9Ya987brsXrLXvd6A/AAAAAMA04egIrrW23lr75sDjLklbJc1ysicAAAAAwPQ0Za7BNcZUSzpT0msjvHy+MWaTMeZRY8yyUfb/sjFmgzFmQ1NT00S2CgAAAACYgqZEwDXGZEv6g6RvWms7j3v5TUlzrLVnSPpXSQ+MdAxr7R3W2jXW2jXFxcUT2i8AAAAAYOpxPOAaY7zqD7e/ttbed/zr1tpOa233wONHJHmNMUWT3CYAAAAAYIpzehVlI+kuSVuttf8yyjZlA9vJGHOO+ntumbwuAQAAAADTgdOrKF8g6VOS3jHGvD1Q+/9JqpIka+1PJH1Y0l8YYxKSIpI+Zq21DvQKAAAAAJjCHA241toXJZmTbPNDST+cnI4AAAAAANOV49fgAgAAAACQDgRcAAAAAEBGIOACAAAAADICARcAAAAAkBEIuAAAAACAjEDABQAAAABkBAIuAAAAACAjEHABAAAAABmBgAsAAAAAyAgEXAAAAABARiDgAgAAAAAyAgEXAAAAAJARCLgAAAAAgIxAwAUAAAAAZAQCLgAAAAAgIxBwAQAAAAAZgYALAAAAAMgIBFwAAAAAQEYg4AIAAAAAMgIBFwAAAACQEQi4AAAAAICMQMAFAAAAAGQEAi4AAAAAICMQcAEAAAAAGeGkAdcYUzgZjQAAAAAAMB5jGcF91RhzrzHmGmOMmfCOAAAAAAA4DWMJuAsl3SHpU5J2GmP+jzFm4cS2BQAAAADAqTlpwLX9nrDW3iTpS5I+I+l1Y8xzxpjzJ7xDAAAAAADGwHOyDQauwf2k+kdwGyT9laSHJK2SdK+kuRPYHwAAAAAAY3LSgCvpFUn/LumD1traY+objDE/mZi2AAAAAAA4NWMJuIustXakF6y13zPG/Ku19q/S3BcAAAAAAKdkTNfgnmSTC9LUCwAAAAAAp20sqygDAAAAADDlEXABAAAAABkhHQHXpOEYAAAAAACMy5gDrjEmNMpLt6WpFwAAAAAATttJA64x5n3GmPckbRt4foYx5kdHXrfW/mLi2gMAAAAAYGzGMoL7fUlXSmqRJGvtJklrJ7IpAAAAAABO1ZimKFtrDx5XSk5ALwAAAAAAnDbPGLY5aIx5nyRrjPFKulnS1oltCwAAAACAUzOWEdyvSvqapFmS6iStGngOAAAAAMCUcdIRXGtts6RPTEIvAAAAAACctpMGXGNMsaQvSao+dntr7ecnri0AAAAAAE7NWK7BfVDSC5KeFItLAQAAAACmqLEE3JC19tsT3gkAAAAAAOMwlkWmHjbGXDPhnQAAAAAAMA5jCbg3qz/kRowxncaYLmNM50Q3BgAAAADAqRjLKsrhyWgEAAAAAIDxOOkIrjHmAmNM1sDjTxpj/sUYUzXxrQEAAAAAMHZjmaL8Y0m9xpgzJH1L0m5J/z6hXQEAAAAAcIrGEnCT1lor6XpJP7TW3i6JacsAAAAAgCllLLcJ6jTG/DdJn5S01hjjGuN+AAAAAABMmrGM4G6XFJX0BWvtYUmVkrImtCsAAAAAAE7RWEZi11hrv3zkibX2gDGmdwJ7AgAAAADglI0acI0xfyHpLyXNM8ZsPualsKSXJroxAAAAAABOxYlGcP9D0qOS/q+kW46pd1lrWye0KwAAAAAATtGoAdda2yGpQ9JNk9cOAAAAAACnZyyLTAEAAAAAMOURcAEAAAAAGcHxgGuMucoYs90Ys8sYc8sIr/uNMb8deP01Y0y1A20CAAAAAKY4RwOuMcYt6XZJV0taKukmY8zS4zb7gqQ2a+0CSd+X9L3J7RIAAAAAMB04PYJ7jqRd1to91tqYpN9Iuv64ba6XdPfA499LWm+MMZPYIwBMiGQspmQslpZjxTo61Ll9u3r271cqHk/LMQEAAKabE90maDLMknTwmOe1ks4dbRtrbcIY0yGpUFLzpHQIAGmWjEbVvmmzDj/2uCSp7MrLlbtypTzBwGkdr3vvXu2+/aeKNjdLxqjsystV8f5r5MnOTmfbAAAAU57TATdtjDFflvRlSaqqqnK4GwAYXee772n3j346+Hz3j+/Q/L/4imSkrm3bFawoV+6K5QqUlp70WMloVLW/v78/3EqStTr82OMKL16s/FUrJ+otAAAATElOB9w6SbOPeV45UBtpm1pjjEdSrqSW4w9krb1D0h2StGbNGjsh3QJAGjS98NKwWuNTTyvZ16feA/2TWkKzZ6nmP98sf0HBCY8V7+xS1/Ydw+qxjg5F6uuV7OmVr7hIvtzc9DQPAAAwhTkdcN+QVGOMmav+IPsxSR8/bpuHJH1G0iuSPizpaWstARbAtOU+fipyKilZq1Q8MVjqPVin3v0HThpwveFsZc2tVvfOXYO1QEW5kl2devfv/l6pWEz+oiLN/+qXlL1gflrfBwAAwFTj6CJT1tqEpK9L+rOkrZJ+Z6191xjzXWPMdQOb3SWp0BizS9J/kTTsVkIAMJ0UX3ShjNt9tOB2K3vRQvXV1w/ZbiyLRbkDAc3+6IflCYcHayWXXKzae+9TamABq2hzs/b98t8V7+5JzxsAAACYopwewZW19hFJjxxX+x/HPO6T9JHJ7gsAJkp40UIt/ptvqX3TZklS7orlan751SHbuEMhhWbPHmn34cerWaCl/+M7ih4+LFcg0B+Uj1tsvvdAreLt7fJmZ6XnTQAAAExBjgdcAJhpjMul8KKFCi9aOFjzFRTIm5uj1tc3KDRntsqvulLB8rIxHzNQXKRAcZEkDY7cHstfWCBPmFWVgclgrVVPZ5uMcSsrh+vfAWAyEXABYAoIlJZo9oc/pPJrrpLL55PLc/pfnrOqq1Wy/hI1PvWMJMl4vZrzqU+w0BQwCXq7OvTehuf13oYX5fF6deZFV2rO4jPV096iaF+vcgqKlZNf5HSbAJCxCLgAMIV4QqHxHyMrpMqP3KjCc89WvLNbwfJSBSoq0tAdgJPZ/e5GbX7laUlSIh7TK4/9QclkUq8/+UdJKfkCQa3/0OdUXl3jbKMAkKEIuACQAWJtbep49z11bd2uUPUc5a1cofDChSffEUDaxGNR7dj02pBaKpVU7a6tysrJUU9nu2J9Eb3y+O91zae+oUCQa+IBIN0IuAAwzSVjMdXd98DR++u+9LJaX31NNd/4mrxMSwYmjcvtVm5BsTpaGgdrqVRKwaywmg7tH6y1Nzeqr6ebgAsAE4CACwDTUKKnV9179ije2iZvXq6aXnxpyOvdu/eo92Ctcgm4wKRxuz1adu4lqtu7Q8lE/22+ssK58gdDiseig9sVllYomBUe7TAAgHEg4E4zh7oatL+tVpI0J69SFTmlDncE4GRsMqnuPXvVu3+/XD6/smsWnNIKycdL9vWp7oGH1PDEk5KkkssuVaqvTy5/YNh5TybR2yubSMibkzPi6/HOTnVu3aae3XsUmDVLucuWyF/EAjnAaMqr5uvaT/2VWhtqZVweFZbN0oEd78i4XLIDo7nnXXGj/MHxX28PABiOgDuN7G+v1Q9evku98YgkKeQN6ubzP6/q/LHdKzOaiKo33qewP1sel3siWwVwjI4t72rHbT+UUilJkq+wQIu+9U0FT3Php0ht3WC4laRI3SGF5sxRpO6QjLv/37a/sEDBylmjHiMVj6t90zuqe+BBJXp6VXrpOhVddIF8eXlHt0kkdOiPf1LDE08N1sILa7Tga38hb+7IgRiAVFQ+W0XlR7835xaWaHbNcsX6ehXOL1J2Tr6D3QFAZiPgTrJoIqq6zgb1xHtVmlWskuzCMe/76sG3BsOtJPXGI3r14FtjCrg7W/bqoa2Pq7bzsFaULtKVNZdoFqO/wIRLRiKqe+ChwXArSbGWVnVu237aATfR3T3kedfWbSped7FyV6xQx5Z3FV5Uo5JL1slfOPrXl66du7Tr9h9L1kqSav9wv+Qyqrj2msFt+uoPq2HgVkOD++3Yqd6DB5Wbu+y0egdmIrfbo8LS0X/hBABIHwLuJOqJRfTw9if19J7+a+VC3qC+es4ntaho/pj2r+usVyKVkCS5jEsu41Jd1+GT7ne4u1E/fPUX6kv0X//zWu3baupt0zfO/ZyCvsBJ9gYwHql4XLG29mH1RGfXaR/TX1YmdzCoZOToL7zaN23Wsr/7jipv/KBcPp+My3XCY3Rv3zEYbo9ofPpZFa+9SN5w/7WBqUR8SDA/IhWPn3bvAAAAE+nEPwEhrfa2HRgMt1L/COxv3nlI3bHek+7b2NOiqrxKdcd61R3rVU+8V0mb1LmVZ55030OdjYPh9og9rfvV2Nt86m8CwJgl+vrUtXuPcpcuUaKnWzZxNBhm1yw47eMGy0q14K/+UoGy/ut4s+bM1oK//Ip8eXlyBwInDbeS5M4avnqrNxyWy3P0957BsjKFFw291ZAnHFZwFiNRAABgamIEdxK19XUMqx3qbFBPrEfZvhMvNrGp/j0d6jys9fMv0Iv7N0jW6pK5F2hl6ZKTnjfg8Q+reVxueV3esTcP4JR1bH5Hu3/0UxWce7aKL7pILa+9Jl9enmb/p48ovLBmXMfOXbpES77zbSW6u+XNyZFnhMB6IjlLl8gTDivRNTCSbIwqrv+A3MHg4DbuYFDVn/mUGp9+Rm1vb1L23GqVXX2VAiXF4+odAABgohBwJ1FRaPiiElW5FQr7s0+6787WPXq3cYfyAjm6bN6FkqTmnhaFfMGT7CnNzi3XkuIabW3aOVi7quYSlWXzQyowUVKJxODiTK2vvSFvQYFK1q1TsKpSReefm5ZzeMPhwenEpypUOUuLv/0tdW3brmSkT9k1C5S9YPjlEsGKclV9/GOq+OB1cgcCQ0Z4AQAAphp+UplEc/Nm6/2LLtMjO55WyqaUF8jRx1Zer5D35CF1cdECbarfqva+Tr104A1J0jULLx3Tashhf7Y+s+pG7Wrdr85op4qzirSgYI5cY5jGCOD0ubxHZ0nEW1vV/NLLKsu6zMGOhgpVVipUWXnS7YzLJW/20F/EJfv61LV9hzq2bJE3L195K5YrVDW2Fd0BAAAmCgF3EgW8AV298BKtKl+mSLxXxVlFyg/mjmnfM8qW6r3GnXqnYZskaX7BnDFdf3tEJBFVZ7RbrZH+gEu4BSaWy+NR2ZWXq3PrtsHFnIzbrfzVZzncWXq0bnxTe+/82eDzw4/9WUtu+a9cnwsAABxFwJ1kHpdbs3PLT3m/wlC+vrD6YzrU1SBrrcrDJco6yXW7RzR2t+hfX/u5WnvbJUlP7n5BN628Xuvmnn/KfQAYu5ylS7ToW99U6+tvyOXzqeDsNcoe57W3U0G8s0t19z84pJbo6lb3rt0EXEw7yURCzfUH1NZUL38wS8WzqpWdk+d0WwCA00TAnUaC3oDmF8yRJLVFOpTo61Ju4OTX3+1tPzAYbo94aNsTOqNs6ZhHkAGcOpfXq9zly5S7PLPuGWuTCaWi0WF1bh+E6Wjv1rf0/B/vkdQ/06J09lxd8sHPKBTm+yMATEcE3Gmmo69TL+x7XU/teUlet0fXLlqvcytXKeAZ/X628eTwHzqjiag6ol3a135Qfrdfs3PLx7TYFYCpK9bermhzizzZ2QqWlY55P2utevcfUM/+A3L7fcqaN++EKyX78vNVetn6IaO4xu1W1tzq8bQPTLqezna9/tRDOhJuJanh4F41Htqv6kUrnWsMAHDaCLjTzMZD7+iP25/sfxKX/mPTA8r1h7WqfPQRotm5FfK4PEqkEoO182afpV+9dZ8Odh6SJC0unq9Pr/qwCkdY6RnA1Ne5fYf2/PROxVrb5Ar4NefjH1Ph+84f06rHnVu3acf3b5ON93+N8BcXa+F/+YaC5aNfTlF88UVyB/xqfPZ5+fLzVX7t1cqaNy9t7weYDIl4TH29PZL6F1MrmTVXHq9H8djwGQoAgOmBgDuNxBIxvbj/jWH1zYe3nTDgzsmr1F+d9zk9uvMZNfW06LzKsyTZwXArSduadmtnyz4CLjANxTo6tPfffq5Ya5skKdUX1d6f/1LBykplz5t7wn2T0agOPfTHwXArSdGmJnW+t+2EAdeXl6eyK69Q8dqLZDyeIStGA9NFVk6eqmqWqq3psKoXn6G9W99SIhZVRfUitTYeUltj//fJovIq5RaWjPm48VhMHq9XxpiJah0AMAoC7jTicrlVGMpXXefhIfWCUN5J911cPF/z86sUS8WVTCX13Wd+MGybpp7mNHUKYDLFW9sVbWoaWrRW0cbGkwbcVCymaOPwf/uxtrYxndsdPPltzpyQ7O1U79531Ltrg3xFlcpaeI58xdzGCEN5vD6tueT9qt29Tc899O8yMvL6A3rtyfu17OyLtfOdN5SIRRUIZenKj31VhWWj31YrGulRb3eXdr3zug7u3qpZcxdq4arzlV9UNonvCADAvWKmEY/LrcvmXySP6+jvJbJ8Ia0sXTKm/b0er7J8IWX5QlpeunjY61W5rH4KOKln7z7t/9V/aOv//Uc1Pvu84h0dY9rPE86WJzz8Gnpvft7gY5tKqa+hUX2HD8smk0e3CYdVdMHwFdXDNQtO/Q1MEdam1PnWk2p98ufq2/eOOjc8qoYHvq94W8NpHzMVj6trx041PvOcWjdsVKy1NY0dw0l5RWXq7elUIJitQChLbo9XyXhMe997U6WV1ZKkvt4e7dqyYcT9Iz1d2vL6s3rwrv+nx3/7U0lSKpHUu68/r+cf/HdFerom660AAMQI7rSzqGie/uuFX9G+9jp5XG7Ny69SRc7YF5ORJLfLrSsWrFVDT5P2tB6Q27h1ec1aLSisnpimAZxUb90hbf9/31eip/96wK7tOxRrvkazPvRBmZPct9pfVKjqT39Su39y52B4Lb/mKoXm9K+6Hu/qUsMTT6n+0T9LqZSKL16r8muvlr+wQFL/9bSJnl41Pfe83MGgKm/8oLIXLZzAdzuxEh3N6nrriSG1VG+nYo375c0/ta+XR7S+sUF77vzZ4D2Nw4sXav5XvizfMb9EwPTl9weG/DtLpVLy+vxKHDN1/8h05ePtfe8tvf7kg0olE4pGetRcf0BnX3q9trz2jFoaDqm9+bCCWSe/4wEAID0IuNNQdf5sVeePb6pdRU6pvnHu59XQ0yyv26Oy7GK5Xe40dQjgVPXu3z8YbmVTsqmU6h99TEVrL1Sg5OTX/uWvPkvL/uffqq+hSd7cHAVnV8oT6F9dveOdd3XooYcHt218+hkFSopUdtWVkiR/UZHmfOJjKrvqChmPR/6C6X8tvj1mVdyjxRFqYxBtbdOBe343ZP+ubTvUs2ePfKvPOt0WMYXMmrdEm195enBxKbfHp7lLz9I7rz0zuM3cpcP/X0f7Inpv44tDi9aqq71Z/mCWopGek/6CCgCQXgRch/TGI3qvcadeObBR+cFcnTf7LFlZbazbLEk6s2KFagqq5ZrAb4xBX0DVvtGvJwIw+WwyoWRPr2wqJZffr579B+TLy5PL5zvhfsblUmj2bIVmD//lV/vbm4bVml95TSWXrR9cZdm43Se8NdB04skpVHjlJUNGcV2BLHlLqk7reKlonxLd3cPqid7e0+4RU0tR+Wxd/Ym/VO3ubYrH+lRWtUCtjYfkMi7J49ayc9Zq9oKlw/ZzudwKBEPqHHjscnuUSibk8fqUTCY0a/5i5RdXTP4bAoAZjIDrkA11m/XrTfcPPn/pwBu6dN6FembvK5Kk5/a9ppvP/7wWF0/f6+AAjF3W3Gp5srL6r5FNpSRJxWsv1IHf/E6+vHyFa+af9rFDlRVqff2481XNlnFn5qwN43Ir56wr5MktVs+21+QvqVLW0gvkKxh9VegT8RUUKHflCnVs2ny06HIpWEFwySRF5VUqKj/6S5DKeYs1d8kqSVI4t2DEkVivz6cz3ne5nvz9XbKSfIGg3B6visurVDKrWpXzl8gfmJoLsQFApiLgOqAn2qM/73xuSK0r1queWI+8Lo/iqYRSNqWXDmwg4AIzRLC8XPO/9lU1PP6Eos0tCtfUKNrSonhLq6JNjQrXzFek/rC6d+5SMtqnrDlzlD1/3phCat5ZZ6nxuRcUa+lfGMkdCqn4kosz+hYmnux85ZxxqcIrLxn3+3T7/ar66IdV63Gr7c235S8sVNVNH1VW9Zw0dYupyLhcyskvOul2FfMW6aqP/4UO798lrz+g8uoaFZYyOwoAnELAdcjw68OGV/oSk3Oj+dbediVtUgXBPK7DBRwULC9TrK1NxuVS0wsvyib6F7jx5ecrcqhe2//5+4MhVS6Xav7qL5V/5qqTHjdUOUuLv/3X6tm7T0pZheZUKVhxeqOZ0026QnxwVoXmf/XLirW2yR0MyJuTk5bjYvpzuz0qn1Oj8jk1TrcCABAB1xFZ/ixdWbNO/3HMFOUsb0jZvizFU0dXbLxwztkT2kck3qeXD2zQH7c9qVgyrgvnnK0ray5WYWj6LzADTEe+/HzN+uD12vWjnw6G27KrrlCoeo5aX339aLiVpFRKdQ88pJzFi8Z0L9pAScmYFqvC6FxerwKl/DcEAGAqI+A65OyKlQp6/HrpwAblB3P1vtmrFUlEtbftoIyR1s+7UEuKJva3wTua9+h3W46urPrcvlcVDmTpA4sun9DzAhhd3qoztPx//q36GhrlzQkrWFkpdyCgRPfwe2nG29uVjMXGFHABAABmAgKuQ0K+oM6pXKVzKlcNqS8r7g+1HvfE/6/Z3rx7WO3VA2/p0rkXKMsXmvDzAxjOGKPgrFkKzpo1pJ41f/giU8VrL5IvN3eyWgNwnOb6A2quPyiXy62iijkqKJkZU/8BYCoj4E4xkxFsjxhpKnJpdpH87hPfjgTA5AvXLND8v/iK6v5wv+Ld3SpZt1bF6y52ui1gxjp8YLf+/JufKpmIS5J8gZCu+vhXVVQ2vvvUAwDGh4A7yRq7W9QaaVeOP1tl4eL+e+xNsJRNaVfLPm0+vFWStLJsiRYUVmtpyUIVhvLV0tsmSfK6vbqqZt2khmwAY+PyelV47tnKWbpEqXhMvvz8jF4FGZjKbCqlrRtfHAy3khTr69X+7e8QcAHAYSSZSbSp/j397M3fqi8Rldfl0U0rr9d5s8+a8JWLd7bs1Q9evksp239vzaf2vKRvvu8LWlQ0X//5fV/U3raDiqcSqsqt0Oxc7usITGXecLbTLQAzXiqVUld787B6d3uLA90AAI418cOHkCQ19bTqF2/dO3jrn3gqoV9tul+HuhokSa2Rdu1pO6DmntYTHea0vLT/jcFwK/WP6L60f4MkqTirUOdUrtIFVWsItwAAjIHb49HCM84bVp+zaKUD3QAAjsUI7iRpj3SoNx4ZUkvZlNoinero69Iv3rpXXdFuZXlD+uSqG3Rm+fJxTz/sS0TV2delnljvsNd6E33jOjYAAE7r7epUIhGTy+1RKDtHLtfk/d6+auEKRSM92vL6c3K53Fp14RUqn7tw0s4PABgZAXeS5AbCCnj8gyO4kuQyLoW8Ad3+2t2D4bcn3qufvfk7fWdticpzSk/7fLUd9frtOw9pZ8s+XVlzsd6sf1det0dG/aH5gqo143tDAAA4pK+nWzs2v6bNLz+pZCKumlXnKSs7V3OXnqmc/KJJ6SGUnaMzLrhCC1acK5fLKJidMynnBQCcGFOUJ0lJdpE+uepD8rj6f6fgMi59ePm1SqQSw0Z248m4mgcWfjodfYmofvfun7SjZa+srLY0bNe1Cy9VVe4sVefP1pfWfFxLiyf2HrsAAEyUAzvf0etPPqCOlgZ1d7TqreceUV+kR+++9qxSqdTJD5BGvoBfTfUH9dIjv9NbLzym5vqDk3p+AMBQjOBOotUVKzQrXKrWSLtyAzkqD5eqobtJHpdHiVRicDuXcSkncPoLybRHOrS9adfg89rOeh3qatBXzv6ElhUvlNfjHdf7AADAKalkUtvffm1YkG2s26tUIqWernaFcwsmrZ992zbrhYfvGXz+7hvP65pPfl0FJaxrAQBOYAR3ErmMSxU5ZVpeulizcyvkcblVHi7RR1e8f3DqsJHRB5dcoVnhspMeL56MK55MDKsHvAHlBsJDaimbkjEuwi0AYFozLpfCeQU6fpmKYFZYxmXk8U7evdz7env01gt/HlKL9UXUcHDPpPUAABiKEVyHuYxLF1SdrTm5lWqNtCkvkKvZueUnvBdtX6JPmw9v01O7X5Tb5dHlCy7SspKF8rn7w2teIEcfXf4B3bXxN4OrJ68oW6y5+ZWT8p4AAJgoxhgtXn2h9m3brHgsKptKyecPKie/RPOXrVYwNHm30kqlUkomhv+iOTXCL58BAJPDWGud7iHt1qxZYzds2OB0GxPmjdpN+reN9wyp/dV5n1VlboV2NO/RgY46zcopU3FWoZp6WpTlDak6v1I5/vAoRwQAYHpprj+gwwf3KJVIKCs3X9k5+SqqqJL7BL8gngibX3lKG555ePC5y+XWNZ/6ukpmVU9qHwAw0xhjNlprh62cywjuFJZMJVXXeVgtkXblB3I0K6dMLuPSc/teHbbt67WbtL15jx7f9fxg7azy5fr0mTcq6A1OZtsAAEy4ovIqFZVXnfJ+kZ4uxWNRZYXz5PaM/8egmpXnyOvza9ubLysrJ0/Lz12n4oo54z4uAOD0EHCnKGutXq19S796+77+62dl9KFlV+uS6vPl9wy/vsjr8uiN2k1Dam/Wb9Gl8y9UTWH1JHUNAMDUlEomdWDnFr3x9IPq7uxQ9eKVOvOiq5RXePq35JP6r/1dsvpCLVhxjlxu16SPIAMAhmKRqSnqcHeTfrP5wcFraK2s7nv3UdV1N+jSee8bXJRKktzGrRVli9XW1zHsOPFkbNJ6BgBgqmo+fFDP3P9LdbW3yaZS2vve23rzuUeVTMTTcnyvz0e4BYApgK/EU1RXtFux5NBvulZWHX1dWl6yUN983xe16fB78rjcWlm2VKVZRSoPl6i+q3Fw+8JQvsrD4/vNNAAAmaCtqV7WDr210P7tm9Xdea1yC4od6goAkG4E3CmqIJinLG9IPfHewZrH5VZRKF9ul1uLi+drcfH8Ift8cfVNemL3C9rWtEsLi+bp8gUXKT+YO9mtAwAw5fgDoWG1QCgsn8/vQDcAgIlCwJ2iirIK9PnV/0k/f/N36o71KODx6xNnfEjl4ZJR96nMLdenVt2oSDyioDcoj8s9iR0DADB1FVdUqWRWtRrr9g1UjM697HoFs3OcbAsAkGbcJmiKa+ltU1ukQzn+sEqyC51uBwCAaauno02Nh/YrGulVXnGZih24rRAAID24TdA0VRjKV2Eo3+k2AACY9rJy8zU3l++pAJDJCLgAAGBGifR0qbXxkFLJpPKLy5VN6AWAjEHABQAAM0ZnW7Oef+jXg9fiZufma/2Hv6DC0lnONgYASAvugwsAAGaM2t1bj1loSuruaNO2jS/KplKj7wQAmDYIuAAAYMZoOXxwWO3wwT1KJOIjbA0AmG4IuAAAYMYoq1owrFZVs1zeNN0Pt7erQwd3vaf9O7aoq70lLccEAIwd1+ACAIAZo2LuItWsPFs7N2+QZFU+Z4Fqzjg3LcfuaGnUM/ffrdbGQ5KkUHaOLv/ol1RYVpmW4wMATo6ACwAAZoyscK7Ov+ojWrpmrVKppHILS+XzB9Jy7Nrd76mnq13Vi8+QtVaH9m7X1jdf1AVXfVTGxaQ5AJgMBFwAADCjeDzeCRlVjfX1aU7Ncu3a8oZcLrdqVp6rvkiPEol42qZAAwBOjF8nAgAApIHH69Wml59QT2e7utpb9Obzj6i4Yg7hFgAmEQEXAADgFPR0tqu18ZCikd7BWjKZ0L7tm+X2+gZrLo9HDbV7nWgRAGYsx6YoG2P+SdIHJMUk7Zb0OWtt+wjb7ZPUJSkpKWGtXTOJbQIAAEiSUqmU9m/fpFcev099Pd0qKqvU+Vd9RMUVVXK53MrOLZDPH5AdCLnG5VI4r8DhrgFgZnFyBPcJScuttSsl7ZD0306w7SXW2lWEWwAA4JTWxjo9++Cv1NfTLUlqPlyrF//0G0UjPTLGaMnqC+R2e2VcbhmXW15fQHOXrHK2aQCYYRwbwbXWPn7M01clfdipXgAAAE6mo6VJNpUaUmtrqld3R5v8wSyVVS3QNZ/6uhrr9snlcqukslqFpdwiCAAm01RZRfnzkn47ymtW0uPGGCvpp9baO0bayBjzZUlflqSqqqoJaRIAAMxcwVD2sJovEJQvEBx8XlwxR8UVcyazLQDAMSZ0irIx5kljzJYR/lx/zDbfkZSQ9OtRDnOhtfYsSVdL+poxZu1IG1lr77DWrrHWrikuLk77ewEAADNbYVmlalaec0zF6NzLPqhwXqFjPQEAhprQEVxr7WUnet0Y81lJ75e03lprRzlG3cDfjcaY+yWdI+n5NLcKAABwQv5gSGevv07zlp6pSE+XcgpKVFg2y+m2AADHcHIV5ask/Y2ki621vaNskyXJZa3tGnh8haTvTmKbAAAAgwLBLM2at9jpNgAAo3ByFeUfSgpLesIY87Yx5ieSZIypMMY8MrBNqaQXjTGbJL0u6U/W2secaRcAAAAAMJU5uYryglHqhyRdM/B4j6QzJrMvAAAAAMD05OQILgAAAAAAaUPABQAAAABkBAIuAAAAACAjEHABAAAAABmBgAsAAAAAyAgEXAAAAABARiDgAgAAAAAyAgEXAAAAAJARCLgAAAAAgIxAwAUAAAAAZAQCLgAAAAAgIxBwAQAAAAAZgYALAAAAAMgIBFwAAAAAQEYg4AIAAAAAMgIBFwAAAACQEQi4AAAAAICMQMAFAAAAAGQEAi4AAAAAICMQcAEAAAAAGYGACwAAAADICARcAAAAAEBGIOACAAAAADICARcAAAAAkBEIuAAAAACAjEDABQAAAABkBAIuAAAAACAjEHABAAAAABmBgAsAAAAAyAgEXAAAAABARiDgAgAAAAAyAgEXAAAAAJARCLgAAAAAgIxAwAUAAAAAZAQCLgAAAAAgIxBwAQAAAAAZgYALAAAAAMgIBFwAAAAAQEYg4AIAAAAAMgIBFwAAAACQEQi4AAAAAICMQMAFAAAAAGQEAi4AAAAAICMQcAEAAAAAGYGACwAAAADICARcAAAAAEBGIOACAABISiVTTrcAABgnj9MNAAAAOCnWFlH3jmb17u9QcFZY2YuL5S8MOd0WAOA0EHABAMCMlexLqPnZveo73C1JirX0qvdAh8qvWyxPls/h7gAAp4opygAAYMaKt0cGw+3RWp9irRGHOgIAjAcBFwAAzFwuM2LZ8BMSAExLfPkGAAAzljcvoKy5eUNqgbJs+Qq4BhcApiOuwQUAADOW2+dRwQVzFJidp766DgXKwgrNyZM76HW6NQDAaSDgAgCAGc0b9it3WYlyl5U43QoAYJyYogwAAAAAyAgEXAAAAABARiDgAgAAAAAyAgEXAAAAAJARCLgAAAAAgIzgWMA1xvxPY0ydMebtgT/XjLLdVcaY7caYXcaYWya7TwAAAADA9OD0bYK+b639f6O9aIxxS7pd0uWSaiW9YYx5yFr73mQ1CAAA4IREb1y9+9vVs7NFvoKgshcVyV+c5XRbADClTfUpyudI2mWt3WOtjUn6jaTrHe4JAABgQllr1bmlQc3P7lWkrlMd7zSo/o/bFGvrdbo1AJjSnA64XzfGbDbG/MwYkz/C67MkHTzmee1ADQAAIGMluqLq2HR4SC0VTSraRMAFgBOZ0IBrjHnSGLNlhD/XS/qxpPmSVkmql/TP4zzXl40xG4wxG5qamsbfPAAAwCRIRuJKRhNOtwEAGWFCr8G11l42lu2MMXdKeniEl+okzT7meeVAbaRz3SHpDklas2aNPbVOAQAAJleyL67uHS39I7Ueo/zVs5Q1N18ur1uesF+5Z5SqfWP94PYuv1v+4pCDHQPA1OfYIlPGmHJr7ZGv2jdI2jLCZm9IqjHGzFV/sP2YpI9PUosAAAATpmdPm1peOjD4vOmpPXJdXaOs6nwZY5SzvEyecEDdO5rlKwwpvLBIvnwCLgCciJOrKP+jMWaVJCtpn6SvSJIxpkLSv1lrr7HWJowxX5f0Z0luST+z1r7rUL8AAGAGs9Yq1tSjaEuvXB6X/KXZ8uYETutYqWRKne82Dqv37m1TVnX/siSekFc5S4qVs6R4XH0DwEziWMC11n5qlPohSdcc8/wRSY9MVl8AAAAjidR26PAjO6VU/5VQ3ryAyq5ZKG/uqYdcY4w8WV7FmofW3SFvOloFgBnL6VWUAQAAprxkLKm21+sGw60kxdv7FDnUdVrHMy6j3JVlksscrXldCs0d6aYSAICxcnKKMgAAwLRg40kluqLD6sne2GkfMzArRxXXL1ZffZeM26VARVj+oqzxtAkAMx4BFwAA4CTcIa+yFxaqY1PDkLq/5PQDqTFGgbKwAmXh8bYHABhAwAUAADgJY4xylpUqFU2qa0eLXD63Cs6rJJwCwBRDwAUAABgDb25ARWurlXdWhYzbJU+2z+mWAADHIeACAACMkXG7TmvVZADA5GAVZQAAAABARiDgAgAAAAAyAlOUAQAAJkkyEle0pVepSELe/IB8hSEZY06+IwBgTAi4AABMAV29Mb2zq1lvvHdYFcXZOmdpmeaU5zjdFtIoGYmr+cX96tnV2l9wGZVesUBZc/OdbQwAMggBFwCAKeCZjQf1h6d3SpLe3N6opzcc1GevXaaG1h7VzM5TTVW+PG6uLJrOoo3dR8OtJKWsml/cJ39pljwhVmQGgHQg4AIA4LDm9oj+9OKewefJlFV9c4+27W/R0xsOykj6+kfP1Jolpc41iXFL9iaG17rjSkWTUsiBhgAgA/GrYAAAHJayVqmUHXyeSKSUskefW0n3P7NTvX1xB7pDunjzht9eyF+aJXcWo7cAkC4EXAAAHFacF9QV51UPPreSgn6P3K6j36a7I3HFEqnJbw5p4yvOUtHF1TI+d//zwpCKLqqWe+A5AGD8mKIMAIDDjDG6/Jw5KsgJ6MW3D6mkIKjC3ICeeuPA4DbrVs9WXrbfwS4xXi6PSzlLSxSclaNULClP2Cd3wOt0WwCQUQi4AABMAXlhv9afXaV1Z1XKStq0s0mbdzarqzemS9bM1tozK51uEWnizR0+VRkAkB4EXAAAphD3wErJqxeXaml1gWKJlHIZuXVUKp5Q5GCnOt9rlNvvUXhpiQIVYe5fCwBTEAEXAIApKhjwKuh0E1DPvg41Pbl78Hn3njaVf2CRghXcpxgAphoWmQIAABhFKp5U56b644pWvQfaHekHAHBijOACAJBGqZTV4ZYeRWIJlRaElB3kFjDT3ghTkZmeDABTEwEXAIA06Ysm9NSGA7r/2V2KJ1KqKsvRF69brjnlTGWdrlxet3LPKFPjE7uPKRoFq/LGtH8qnlC8PSoZyZsXlMvD5DkAmEgEXAAA0mRXbYd+9+SOwecHDnfq3qd36K8+ukp+L99yp6vQnDyVXlWjrm1NcvndCi8qVqAs+6T7xTv61PLyAfXua5eMFF5cpPyzK+XJYlQfACYK320BAEiTw609cruMcrJ86uyJKZmyend3izq6YyrJ51vudOXyupU1N19Zc/NPab+uHS394VaSrNS1tVn+0mzlLCmRtVbx9j6lYgl5wn55QoReAEgHvtsCAJAmxXlBXXxWpRpae7VqYUiRvrjqmnuUFfA63RomWSqRVO+e1mH1SG2nshcUqmtrk1pfq5VNpOTN8av4svkKlJ58VBgAcGIEXAAA0qAvmtBLm+r09IZaxRJJSdLyeYX69LVLlRUk4GaiRCSuRGdULr9b3tzAkIWnjNulQHlYsdbIkH38xVmKNfeo5aUDg7V4Z1TNz+9T+QcWyx0Y/qOZTaYUa+/rD8O5gRG3AQD04yskAABpUNfcrdfePaxgwCNf0i0rqwMNXU63hQnS19Ctpqd3K94elfG4VHD+bIUXFcnldUvqX2U5vLRYvfvblOiOS5J8hUGFqvMUPdw97Hix5l4le+PDwmsyElf7psPq2HRYSln5S7JUfMlc+QpCE/8mAWAaIuACAJAGyURKkuQykstjJPWP5iUG6sgcyWhCLS/s618dWZJNpNTywn75C0MKlIcHt/MXZan8g0sVa+mVMUa+opA8WT4lumLDjunJ8ckVcA+r9x3uVsdbR+/DG23sUfvb9Sq+eK6MmxWZAeB4fGUEACANyouyVFU29HZAJfkhzSrhuspMk+yJK9rUO6we7+gbVvOG/cqqzldoTt7g6sn+4iyFlxYPbmM8LhVdVD3iQlPR5p7+B9bKplKyqZR697cr2ZdI07sBgMzCCC4AAGkQzvLrKzes0FNvHNSW3U1aXF2oy8+pUn444HRrSDNXwC132KfkcSOx7tDYrrV2BzwqOH+2smuKlOqLy5MXkH+UKcfe3IBkrVLRhGzKSpL8pdlKJZkZAAAjIeACAJAmlSVhferqJYpEaxTwe+R2mZPvhGnHE/Kp+MI5anh8l2yyP3SGFxfJX5w15mO4fR4FK8In3S5YEZa/LFu9e9skSa6AR6HZuYrsa5dvZdnpvQEAyGAEXAAA0sjlMqyaPAME5+Sp4sZlirf3yR3wyFcUktuf/h+r3CGffEUh+UuypZSVTVl1bG5QsDJHuQRcABiGgAsAAHCKjDHyF4bkL5zY1YyNy8jl8wxZaErSkMWsAABHscgUAADAFBZeWCRfQXDweaAsW1lz8x3sCACmLkZwAQAApjBfQVBlH1ikeGtEMka+gqDcTIMHgBERcAEAAKY4T8g34m2EAABDMUUZAAAAAJARGMEFAABwWCqeVKy5R/HOmDxhn/zFIbm8/JgGAKeKr5wAAAAOssmUOt9pUOtrtYO1/LMrlLuqQi4Pk+0A4FTwVRMAAOAUJPviirVFlIzE03K8WFtErW/UDam1bTikWFskLccHgJmEEVwAADBjWGtljDnt/SP1nWp5fr9irRF58wMquqhawVk54+op1ZeQUva4RqVUmgI0AMwkjOACAICMFznUqcYndqn+wa3q2tZ0WqOv8a6oGv+8S7HW/pHVeFufGv68U/GOvnH15snxyxUYOuZgfG55cwPjOi4AzEQEXAAAkNH6Grp1+OHt6t7Vqr76bjU9s1fdu1pP+TiJjqiSkcSQWiqaVLxzfAHXmxNQ6RXz5c3pvw2QJ8en0isWEHAB4DQwRRkAAGS0vvou2eTQKcAdb9cre0GB3EHvmI/jCrglI+m42cQu//h/nArOylX5DUuV7I3LHfLKJlKK1HXKHfTImx8c17RqAJhJCLgAACCzuUYIh8b0h9UxSiWScoe8yltdofYNhwbruWeWy5cfTEOTkifkkyfkU++BdjU+uVupaFLGbVR44RyFFxXJuJl4BwAnQ8AFAAAZLVgelvG4ZBOpwVr+6gq5A2MbvY3Udqr9rUOKd/Qp54wylV5do1QkIXfYJ39xllxed9p6jXX0KVLXqeyaQkXquxRviaj5+X3yF2fJX5yVtvMAQKYi4AIAgIzmL85S+QcWq3tXixI9MYVrChWoHNvKx9GmHh1+ZPvgFOfWFw8oZ1mJCi+cIzPSyPA4xDujan3loLrebZCMlDW/UN55+erd06Z4V5SACwBjQMAFAAAZL1CWrUBZ9invF23uGXb9bufWJuWuKpM3J72LQPXsaVXvnoHFr6zUs6tFeatnyXhc8mT50nouAMhUXMwBAAAwCuMZ/qOSy+tK++ittVY9u1slY4YsWhVt7FbhhXPkKwyl9XwAkKkIuAAAAKMIlGQP3r7niPxzKuXJ9qf1PMYYBcr7R5iN28gd8Mrl9yhYlafwwkK5RgjaAIDhmKIMAAAwCm9uQKXXLFKktlOJrqgCFWEFKsITcq7sRcXq2d2mRHdMckm+3KDCC1k9GQBOBQEXAADgBHz5wbTdCuhE/IUhlX9wiWItvTKSfEWhtI8UA0CmI+ACAIAZq6+hW93bmxVv71P2wkKFqvPGfPugieAN++UNE2oB4HQRcAEAwIwUbelV/R+3ycb7748bqetUwbmVyjurwuHOAACni4s6AADAjBRt7BkMt0e0v1WvRHfUoY4AAONFwAUAADgivXf/AQBMMgIuAACYkfwlWTI+95Ba3qpyFnYCgGmMa3ABAMCM5C8Mqfz9i9S9o1nxtqOLTAEApi8CLgAAmLECpdkKlGY73QYAIE2YogwAAAAAyAgEXAAAAABARnBsirIx5reSFg08zZPUbq1dNcJ2+yR1SUpKSlhr10xSiwAAAACAacSxgGut/U9HHhtj/llSxwk2v8Ra2zzxXQEAAAAApivHF5kyxhhJH5V0qdO9AAAAAACmr6lwDe5FkhqstTtHed1KetwYs9EY8+VJ7AsAAAAAMI1M6AiuMeZJSWUjvPQda+2DA49vknTPCQ5zobW2zhhTIukJY8w2a+3zI5zry5K+LElVVVXj7BwAAAAAMN0Ya61zJzfGI6lO0mprbe0Ytv+fkrqttf/vRNutWbPGbtiwIT1NAgAAAACmFGPMxpEWIHZ6ivJlkraNFm6NMVnGmPCRx5KukLRlEvsDAAAAAEwTTgfcj+m46cnGmApjzCMDT0slvWiM2STpdUl/stY+Nsk9AgAAAACmAUdXUbbWfnaE2iFJ1ww83iPpjEluCwAAAAAwDTk9ggsAAAAAQFoQcAEAAAAAGYGACwAAAADICI5egwsAAOCEeFdU8baIjNctX0FQbj8/EgFAJuCrOQAAmFH6GrrV8OgOJSMJSVL2wkIVnD9bnpDP4c4AAONFwAUAADNGKp5U2+u1g+FWkrp3tChUna/s+QWj7hfv6FP3rlb17m9TcHaushcUypcfHPM5XV73uHsHAJwcARcAAMwYqWhCfQ3dw+qJrr5R90n2JdT87F5FDnVJkqINPerd166yaxfJE/KOul+0tVfdW5vUW9up0JxchRcVyZcfGv+bAACMikWmAADAjOEKeBSclTOs7s0NjLpPvC0yGG6PiDX3Kt7WO+o+id6Ymp7YrY7NDYq3RtTx1mE1Pb1Xyb746TcPADgpAi4AAMgo1lrFO/oUa4vIJlNDXnN53MpfM0veHH9/wUg5K0oVKA+PfkBzyi8o3hpRrDUypBZt7FGsLTLKHgCAdGCKMgAAyBjJaEJd7zaqbeMh2WRK2TWF/YH2mBFaf3GWym9Yonh7n1xet7z5Qbk8o//O35sfVHB2jiIHO48eozRLvoITXIPrGjn8GjN6KAYAjB8BFwAAZIy+uk61vlY7+Lx7R4s82T4VnDt7yHaekG/Mqya7/R4VXVStnr1tihzsUGBWjrLm5csdHP36W19BUMFZYUXqjk5tDlXnyVvANbgAMJEIuAAAIGNE6ruG1bp3tij3jDK5A6MH0pPx5gaUt6pceavKx7S9O+BV0bq56t3fob7DXQpUhBWqypPbx2rKADCRCLgAACBjeHOGLxblKwjKeCY/WHpzAspdEVDuitJJPzcAzFQsMgUAADJGcHaOvHlHQ67xupR3ZsUJr7EFAGQORnABAEDG8OUFVXbtIsWaepRKpuQrCsnPda8AMGMQcAEAQEbx5viP3gYIADCjMF8HAAAAAJARCLgAAAAAgIxAwAUAAAAAZAQCLgAAAAAgIxBwAQAAAAAZgYALAAAAAMgIBFwAAAAAQEYg4AIAAAAAMgIBFwAAAACQEQi4AAAAAICMQMAFAAAAAGQEAi4AAAAAICMQcAEAAAAAGYGACwAAAADICARcAAAAAEBGIOACAAAAADICARcAAAAAkBEIuAAAAACAjEDABQAAAABkBAIuAAAAACAjGGut0z2knTGmSdJ+p/vAmBRJana6CUxpfEZwMnxGcDJ8RnAyfEZwInw+pqY51tri44sZGXAxfRhjNlhr1zjdB6YuPiM4GT4jOBk+IzgZPiM4ET4f0wtTlAEAAAAAGYGACwAAAADICARcOO0OpxvAlMdnBCfDZwQnw2cEJ8NnBCfC52Ma4RpcAAAAAEBGYAQXAAAAAJARCLgAAAAAgIxAwIUjjDEfMca8a4xJGWPWHFOvNsZEjDFvD/z5iZN9wjmjfUYGXvtvxphdxpjtxpgrneoRU4cx5n8aY+qO+dpxjdM9wXnGmKsGvk7sMsbc4nQ/mHqMMfuMMe8MfN3Y4HQ/cJ4x5mfGmEZjzJZjagXGmCeMMTsH/s53skecGAEXTtki6UOSnh/htd3W2lUDf746yX1h6hjxM2KMWSrpY5KWSbpK0o+MMe7Jbw9T0PeP+drxiNPNwFkDXxdul3S1pKWSbhr4+gEc75KBrxvc5xSS9Av1/3xxrFskPWWtrZH01MBzTFEEXDjCWrvVWrvd6T4wdZ3gM3K9pN9Ya6PW2r2Sdkk6Z3K7AzANnCNpl7V2j7U2Juk36v/6AQCjstY+L6n1uPL1ku4eeHy3pA9OZk84NQRcTEVzjTFvGWOeM8Zc5HQzmHJmSTp4zPPagRrwdWPM5oHpZUwfA18rMBZW0uPGmI3GmC873QymrFJrbf3A48OSSp1sBifmcboBZC5jzJOSykZ46TvW2gdH2a1eUpW1tsUYs1rSA8aYZdbazglrFI45zc8IZqgTfV4k/VjS36v/h9W/l/TPkj4/ed0BmKYutNbWGWNKJD1hjNk2MIIHjMhaa40x3Gd1CiPgYsJYay87jX2ikqIDjzcaY3ZLWiiJhR8y0Ol8RiTVSZp9zPPKgRoy3Fg/L8aYOyU9PMHtYOrjawVOylpbN/B3ozHmfvVPbSfg4ngNxphya229MaZcUqPTDWF0TFHGlGKMKT6yYJAxZp6kGkl7nO0KU8xDkj5mjPEbY+aq/zPyusM9wWEDP3AccYP6FynDzPaGpBpjzFxjjE/9i9M95HBPmEKMMVnGmPCRx5KuEF87MLKHJH1m4PFnJDHLbApjBBeOMMbcIOlfJRVL+pMx5m1r7ZWS1kr6rjEmLikl6avW2uMv9McMMNpnxFr7rjHmd5Lek5SQ9DVrbdLJXjEl/KMxZpX6pyjvk/QVR7uB46y1CWPM1yX9WZJb0s+ste863BamllJJ9xtjpP6fif/DWvuYsy3BacaYeyStk1RkjKmV9HeSbpX0O2PMFyTtl/RR5zrEyRhrmUIOAAAAAJj+mKIMAAAAAMgIBFwAAAAAQEYg4AIAAAAAMgIBFwAAAACQEQi4AAA4zBiTZ4z5S6f7AABguiPgAgDgvDxJBFwAAMaJgAsAgPNulTTfGPO2MeZOY8zzA4+3GGMukiRjTLcx5n8bYzYZY141xpQO1IuNMX8wxrwx8OeC0U5ijLnNGPM/Bh5fOXAefhYAAGQM7oMLAIDDjDHVkh621i43xnxLUsBa+7+NMW5JIWttlzHGSrrOWvtHY8w/Suq01v6DMeY/JP3IWvuiMaZK0p+ttUtGOU9I0huSvi7pJ5Kusdbunoz3CADAZPA43QAAABjiDUk/M8Z4JT1grX17oB6T9PDA442SLh94fJmkpcaYI/vnGGOyrbXdxx/YWttrjPmSpOcl/WfCLQAg0zAtCQCAKcRa+7yktZLqJP3CGPPpgZfi9ui0q6SO/pLaJek8a+2qgT+zRgq3x1ghqUVSxQS0DwCAowi4AAA4r0tSWJKMMXMkNVhr75T0b5LOOsm+j0v6qyNPjDGrRttw4NjfknSmpKuNMeeOr20AAKYWpigDAOAwa22LMeYlY8wWSVmSeowxcUndkj594r31DUm3G2M2q//7+vOSvnr8RqZ/DvNdkv7aWnvIGPMF9Y8Qn22t7Uvn+wEAwCksMgXg/9+uHdoADMNAFLVHLS7IlJ3L4RmgkU7vQSPTLx0AAEQwUQYAACCCiTIAhOnup6rWcf5m5r3xDwD8xUQZAACACCbKAAAARBC4AAAARBC4AAAARBC4AAAARBC4AAAARBC4AAAARNgQM/fgB9xkwwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1152x720 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "tsne(tsne_input, 'hue')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a31814a8",
   "metadata": {
    "id": "a31814a8"
   },
   "source": [
    "FastRP embeddings and the Louvain algorithm were executed independently, and yet, we can observe that FastRP embeddings cluster nodes in the same community close in the embedding space. This is no surprise as FastRP is a community-based node embedding algorithm, meaning that nodes close in the graph will be also close in the embedding space. Next, we will evaluate the cosine similarity between nodes in the graph."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "uItwHO-c2WRT",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81
    },
    "id": "uItwHO-c2WRT",
    "outputId": "ee004136-2a40-47e1-b73b-29b2b36a400a"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>nodeProjection</th>\n",
       "      <th>relationshipProjection</th>\n",
       "      <th>graphName</th>\n",
       "      <th>nodeCount</th>\n",
       "      <th>relationshipCount</th>\n",
       "      <th>projectMillis</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>{'Character': {'label': 'Character', 'properti...</td>\n",
       "      <td>{'__ALL__': {'orientation': 'NATURAL', 'indexI...</td>\n",
       "      <td>knn</td>\n",
       "      <td>126</td>\n",
       "      <td>549</td>\n",
       "      <td>23</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                      nodeProjection  \\\n",
       "0  {'Character': {'label': 'Character', 'properti...   \n",
       "\n",
       "                              relationshipProjection graphName  nodeCount  \\\n",
       "0  {'__ALL__': {'orientation': 'NATURAL', 'indexI...       knn        126   \n",
       "\n",
       "   relationshipCount  projectMillis  \n",
       "0                549             23  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "run_query(\"\"\"\n",
    "CALL gds.graph.project('knn', 'Character', '*', {nodeProperties:['embedding']})\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "fL9bP3522Ym-",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "fL9bP3522Ym-",
    "outputId": "e0e76b87-0f1e-4d48-f48e-38e804cbf82b"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'p1': 0.4658985137939453,\n",
       " 'max': 1.0000057220458984,\n",
       " 'p5': 0.49489784240722656,\n",
       " 'p90': 0.9789714813232422,\n",
       " 'p50': 0.7059078216552734,\n",
       " 'p95': 0.9907131195068359,\n",
       " 'p10': 0.5113391876220703,\n",
       " 'p75': 0.8725299835205078,\n",
       " 'p99': 0.9972286224365234,\n",
       " 'p25': 0.5559902191162109,\n",
       " 'p100': 1.0000057220458984,\n",
       " 'min': 0.40580177307128906,\n",
       " 'mean': 0.7212781183151972,\n",
       " 'stdDev': 0.17346463743052318}"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "run_query(\"\"\"\n",
    "CALL gds.knn.stats('knn', {nodeProperties:'embedding', topK: 1000, similarityCutoff:0.1})\n",
    "YIELD similarityDistribution\n",
    "\"\"\")['similarityDistribution'][0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7b99752",
   "metadata": {
    "id": "c7b99752"
   },
   "source": [
    "An average cosine similarity coefficient between all nodes in the graph is around 0.7. Nodes are so similar in the embedding space because we have a tiny graph of only 126 nodes. Next, we will evaluate the cosine and euclidian distance between pairs of nodes connected by a relationship."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "e89d4c7d",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 386
    },
    "id": "e89d4c7d",
    "outputId": "4fff9425-7718-4839-9b32-74907f20bb8a"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<seaborn.axisgrid.FacetGrid at 0x7f23ea906bb0>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAsgAAAFgCAYAAACmDI9oAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAbeElEQVR4nO3de7BlV10n8O8v6QCOiQQkk4lN9wQxyuCDgCFgwKlo0InUSGBEglIQEE1UYBRHa1BLoXxUYcnoCIxABCRQDAQRNGYQjIggRCEh5kV4ZURMJ5EEUB4iME1+88fdDYvmdvfpx7nnnnM/n6pTd5+91z3nt+7pXv3tddfeu7o7AADAmqMWXQAAAGwmAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBAZiVU1alV9fD9HD+tqp67kTUdrKo6s6ounbYfUVXP2Ee7T09fv66qXruRNQJby7KPrVX141X1hEXXwfIp10FmFVTVE5Oc1t1PXefYtu7evfFVHZyqOjPJz3b3fz5Au09397EbUhSwpa3C2AqHwgwym0JVnVxV76uql1XVB6rqlVX1sKp6R1V9sKpOn9p9dVW9tKreVVV/W1XnVNWdkvxKknOr6uqqOreqnlVVr6iqdyR5xV6zs8dW1e9X1XVVdW1V/cBh1n50Vf1mVV0xvd4F0/4vvuf0/PnTPzapqgdW1eVVdc3Ul+P2es0nVtXzp+17VdVfT/X+2l4/s+uH7b+qqqumxxlDDX9ZVa+dfr6vrKo6nP4Cy2MFxtbnVNX10+s9bdp/1lTjdVPNd572P7uqbpjaPmfa96yq+tlp+y+r6jemPn6gqr5zeJ+vGMPZ2rYtugAYfEOSH0zyI0muSPLDSR6a5BFJfiHJI5P8YpK/6O4fqarjk7wryZ8n+eUMsxxV9awk903y0O7+12l2do9fSvKJ7v7Wqe3d9i6kqn47yXetU+Oru/vZe+178vR6D5wG6ndU1Z/tq5PTPzoXJzm3u6+oqq9J8q/7ap/kd5K8oLtfXlVP2Ueb25J8T3d/tqpOSfKqJKdNx+6f5JuT3JLkHUkekuTt+3k/YLUs69h6fpKTk5za3bur6u5VdZckL0tyVnd/oKpenuQnquoVSR6V5D7d3VMf1rOtu0+vtWUjz0zysOxjDO/uD+3jNdgCBGQ2kw9193VJUlXvSfLmaaC7LmuDZJJ8b5JH7JkRSHKXJDv38XqXdPd6wfNhSR6750l3/9PeDbr76QdR9/cm+baqevT0/K5JTkny+X20/6Ykt3b3FdN7fTJJ9jOx+5Ake2ZiXpHkN9Zpc0yS51fVqUm+kOQbh2Pv6u5d03tcnbWfpYAMW8eyjq0PS/LCPcs4uvvjVXW/qT8fmNpclOQpSZ6f5LNJXjLNaF+63gsmed309d358r6vN4YLyFuYgMxm8rlh+47h+R350p/VSvID3f3+8Rur6kHrvN6/HGohBznLUUme1t1v2us1HpovX8Z0l0OtJ8mBThZ4epKPJLnf9J6fHY6NP9cvxN972GqWdWyd2TTDfHqSs5I8OslTk3z3Ok339H0cC9cdw9narEFm2bwpydP2rKOtqvtP+z+V5Lh9fteXuyxrMw6ZXuMrfg3Y3U/v7lPXeaw3gL8pa7/iO2Z6vW+sqq9O8uEk962qO0+/7jtrav/+JCdV1QOn9sdV1f5C6zvypVmZx+2jzV2zNit9R5LHJzl6P68HsLfNOLZeluSCPeNjVd09a+PnyVX1DVObxyd5a1Udm+Su3f2GrE0Y3G/GmpN9j+FsYQIyy+ZXs7ac4NrpV4W/Ou1/S9bC6NVVde4BXuPXktxtOvHjmqw/m3EwXpzkhiRX1dpJcy/K2jq3m5K8Jsn109e/TZLu/nySc5M8b3r/y7L/2eWfSvKU6deh2/fR5neTnDe93n1yGDM8wJa0WcfWf5hquibJD3f3Z5M8KckfTGPiHUlemLUQf2lVXZu1JWQ/c5Dv8xVj+GHWzpJzmTcAABiYQQYAgIGADAAAAwEZAAAGAjIAAAyW+izNs88+u9/4xjcuugyARTrkW4cbQwHWH0OXegb5ox/96KJLAFhaxlCA9S11QAYAgCNNQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAsge07dqaqZnps37Fz0eXCUtu26AIAgAO7ZddNOfdFl8/U9uILzphzNbDazCADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgAwDAQEAGAICBgAwAAAMBGQAABgIyAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGcwvIVbWjqt5SVTdU1Xuq6qem/Xevqsuq6oPT17tN+6uqnltVN1bVtVX1gHnVBgAA+zLPGeTdSf5bd983yYOTPKWq7pvkGUne3N2nJHnz9DxJvi/JKdPj/CQvmGNtAACwrrkF5O6+tbuvmrY/leS9SbYnOSfJRVOzi5I8cto+J8nLe83fJDm+qk6aV30AALCeDVmDXFUnJ7l/kncmObG7b50O/WOSE6ft7UluGr5t17QPAAA2zNwDclUdm+QPk/x0d39yPNbdnaQP8vXOr6orq+rK22+//QhWCrD6jKEABzbXgFxVx2QtHL+yu1837f7InqUT09fbpv03J9kxfPs9p31fprsv7O7Tuvu0E044YX7FA6wgYyjAgc3zKhaV5CVJ3tvdvzUcuiTJedP2eUn+eNj/hOlqFg9O8olhKQYAAGyIbXN87YckeXyS66rq6mnfLyR5dpLXVNWTk3w4yWOmY29I8vAkNyb5TJInzbE2AABY19wCcne/PUnt4/BZ67TvJE+ZVz0AADALd9IDAICBgAwAAAMBGQAABgIyAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgAwDAQEAGAICBgAwAAAMBGQAABgIyAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAFbNUdtSVQd8bN+xc9GVwqa0bdEFAABH2B27c+6LLj9gs4svOGMDioHlYwYZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgAwDAQEAGAICBgAwAAAMBGQAABnMLyFX10qq6raquH/Y9q6purqqrp8fDh2M/X1U3VtX7q+o/zasuAADYn3nOIL8sydnr7P/t7j51erwhSarqvkkem+Sbp+/53ao6eo61AQDAuuYWkLv7bUk+PmPzc5K8urs/190fSnJjktPnVRsAAOzLItYgP7Wqrp2WYNxt2rc9yU1Dm13Tvq9QVedX1ZVVdeXtt98+71oBVooxFODANjogvyDJvZOcmuTWJP/jYF+guy/s7tO6+7QTTjjhCJcHsNqMoQAHtqEBubs/0t1f6O47kvxevrSM4uYkO4am95z2AQDAhtrQgFxVJw1PH5VkzxUuLkny2Kq6c1XdK8kpSd61kbUBAECSbJvXC1fVq5KcmeQeVbUryTOTnFlVpybpJH+f5IIk6e73VNVrktyQZHeSp3T3F+ZVGwAA7MvcAnJ3/9A6u1+yn/a/nuTX51UPAADMwp30AABgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgAwDAQEAGAICBgAwAAIOZAnJVPWSWfQAAsOxmnUF+3oz7AABgqW3b38Gq+o4kZyQ5oap+Zjj0NUmOnmdhAACwCPsNyEnulOTYqd1xw/5PJnn0vIoCAIBF2W9A7u63JnlrVb2suz+8QTUBAMDCHGgGeY87V9WFSU4ev6e7v3seRQEAwKLMGpD/IMkLk7w4yRfmVw4AACzWrAF5d3e/YK6VAADAJjDrZd7+pKp+sqpOqqq773nMtTIAAFiAWWeQz5u+/tywr5N8/ZEtBwAAFmumgNzd95p3IQAAsBnMFJCr6gnr7e/ulx/ZcgAAYLFmXWLxwGH7LknOSnJVEgEZAICVMusSi6eNz6vq+CSvnkdBAACwSLNexWJv/5LEumQAAFbOrGuQ/yRrV61IkqOT/Ickr5lXUQAAsCizrkF+zrC9O8mHu3vXHOoBAICFmmmJRXe/Ncn7khyX5G5JPj/PogAAYFFmCshV9Zgk70ryg0kek+SdVfXoeRYGAACLMOsSi19M8sDuvi1JquqEJH+e5LXzKgwAABZh1qtYHLUnHE8+dhDfCwAAS2PWGeQ3VtWbkrxqen5ukjfMpyQAAFic/QbkqvqGJCd2989V1X9J8tDp0F8neeW8iwMAgI12oBnk/5nk55Oku1+X5HVJUlXfOh37/jnWBgAAG+5A64hP7O7r9t457Tt5LhUBAMACHSggH7+fY191BOsAAIBN4UAB+cqq+rG9d1bVjyZ593xKAgCAxTnQGuSfTvL6qnpcvhSIT0typySPmmNdAACwEPsNyN39kSRnVNV3JfmWaff/6e6/mHtlAACwADNdB7m735LkLXOuBQAAFs7d8AAAYCAgAwDAQEAGAIDB3AJyVb20qm6rquuHfXevqsuq6oPT17tN+6uqnltVN1bVtVX1gHnVBQAA+zPPGeSXJTl7r33PSPLm7j4lyZun50nyfUlOmR7nJ3nBHOsCAIB9mltA7u63Jfn4XrvPSXLRtH1RkkcO+1/ea/4myfFVddK8agMAgH3Z6DXIJ3b3rdP2PyY5cdrenuSmod2uad9XqKrzq+rKqrry9ttvn1+lACvIGApwYAs7Sa+7O0kfwvdd2N2ndfdpJ5xwwhwqA1hdxlCAA9vogPyRPUsnpq+3TftvTrJjaHfPaR8AAGyojQ7IlyQ5b9o+L8kfD/ufMF3N4sFJPjEsxQAAgA0z062mD0VVvSrJmUnuUVW7kjwzybOTvKaqnpzkw0keMzV/Q5KHJ7kxyWeSPGledQEAwP7MLSB39w/t49BZ67TtJE+ZVy0AADArd9IDAICBgAwAAAMBGQAABgIyAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADAXkBtu/Ymaqa6bF9x85FlwsAsKVsW3QBW9Etu27KuS+6fKa2F19wxpyrAQBgZAYZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwEBABgCAgYAMAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgH0Hbd+xMVR3wAQDA5rVt0QWsklt23ZRzX3T5AdtdfMEZG1ANAACHwgwyAAAMBGQAABgIyAAAMBCQAQBgICBvdkdtm+nKGNt37Fx0pQAAK8FVLDa7O3a7MgYAwAYygwwAAAMBGQAABgIyAAAMBGQAABgIyAAAMBCQAQBgICADAMBAQAYAgIGADAAAAwEZAAAGAjIAAAy2LeJNq+rvk3wqyReS7O7u06rq7kkuTnJykr9P8pju/qdF1AcAwNa1yBnk7+ruU7v7tOn5M5K8ubtPSfLm6TkAAGyozbTE4pwkF03bFyV55OJKAQBgq1pUQO4kf1ZV766q86d9J3b3rdP2PyY5cb1vrKrzq+rKqrry9ttv34hal8NR21JVMz2279i56GqBBTGGAhzYQtYgJ3lod99cVf82yWVV9b7xYHd3VfV639jdFya5MElOO+20ddtsSXfszrkvunymphdfcMaciwE2K2MowIEtZAa5u2+evt6W5PVJTk/ykao6KUmmr7ctojYAALa2DQ/IVfXVVXXcnu0k35vk+iSXJDlvanZekj/e6NoA4EjYvmOnZW+wxBaxxOLEJK+vqj3v/7+7+41VdUWS11TVk5N8OMljFlAbABy2W3bdZNkbLLEND8jd/XdJ7rfO/o8lOWuj69mSphP6DuTr7rkjN9/0DxtQEADA5rGok/RYpBlP6DOrAQBsRZvpOsgAALBwAjIAAAwEZAAAGAjIAAAwcJIeACzSjFcWAjaOgAwAi+TKQrDpWGIBAAADARkAAAYCMgAADARkAAAYCMgAADAQkAEAYCAgAwDAQEA+gO07dqaqZnoAALD83CjkAG7ZddNMF3BPXMQdAGAVmEEGAICBgAwAAAMBGQAABgIy+3bUtplOTty+Y+eiKwUAOGKcpMe+3bF7phMUnZwIAKwSM8gAADAQkAEAYCAgAwDAQEAGAICBgAwAAAMBGQAABgIyAAAMBGQAABgIyAAAMBCQAQBgICDDYdi+Y2eqaqbH9h07F10uADCDbYsugBVw1LZU1UxNv+6eO3LzTf8w54I2zi27bsq5L7p8prYXX3DGnKsBAI4EAZnDd8duIREAWBmWWLD0Zl3mYIkDADALM8gsvVmXORzM7PX2HTtzy66bDqcsAGBJCciwjnmEbgBgOQjIbB0HcTIhALB1CchsHU4mBABm4CQ9AJjRrCcFA8vNDDIAzMj5CbA1mEEGAICBgAzAQVm1W6wfTH+ArcESC9goM15FY9Vux72VHcz1tJfpc1+1W6yvWn+Awycgw0aZ8Soa/gFeHYIXwHKyxAIAAAYCMhtrWmawCusWN4NZ104u8ue5autVAVh9lliwsWZdZvAT/9EJMTNYhktOWWbAkTbr2u6jj7lzvvD/PrcBFQGrRkBmc9rKd72bxy2xD+I1l+lkMVbHwZzQmGTm/xhu9v9AApuTgAybzTz+c7CV/8PBUvCbBmAzsQYZWGnLsE4bgM3FDDLw5Vbses3LsE4bgM1lywbkg13vBluG6zUDsMVtuoBcVWcn+Z0kRyd5cXc/ex7vY1YJDtM8TiYEgE1gUwXkqjo6yf9K8j1JdiW5oqou6e4bFlsZ8BXMNAOwojbbSXqnJ7mxu/+uuz+f5NVJzllwTcAmczA3H5nZjDexqapsu9Ndjux7A7CpVHcvuoYvqqpHJzm7u390ev74JA/q7qcObc5Pcv709JuSvH/Gl79Hko8ewXI3k1XuW7La/dO35bVZ+vfR7j571saHOIZulr7Oyyr3T9+W1yr3bzP1bd0xdFMtsZhFd1+Y5MKD/b6qurK7T5tDSQu3yn1LVrt/+ra8lrV/hzKGLmtfZ7XK/dO35bXK/VuGvm22JRY3J9kxPL/ntA8AADbEZgvIVyQ5paruVVV3SvLYJJcsuCYAALaQTbXEort3V9VTk7wpa5d5e2l3v+cIvfxBL8tYIqvct2S1+6dvy2vV+zda9b6ucv/0bXmtcv82fd821Ul6AACwaJttiQUAACyUgAwAAIOVC8hVdXZVvb+qbqyqZ6xz/M5VdfF0/J1VdfICyjwkM/TtiVV1e1VdPT1+dBF1HoqqemlV3VZV1+/jeFXVc6e+X1tVD9joGg/VDH07s6o+MXxuv7zRNR6qqtpRVW+pqhuq6j1V9VPrtFnmz26W/i3t57c346fxc7NZ5fEzWe0xdOnHz+5emUfWTuz7v0m+PsmdklyT5L57tfnJJC+cth+b5OJF130E+/bEJM9fdK2H2L//mOQBSa7fx/GHJ/nTJJXkwUneueiaj2Dfzkxy6aLrPMS+nZTkAdP2cUk+sM6fy2X+7Gbp39J+fnv1w/hp/Nx0j1UeP6f6V3YMXfbxc9VmkGe5VfU5SS6atl+b5Kyqpbgn7Erfhru735bk4/tpck6Sl/eav0lyfFWdtDHVHZ4Z+ra0uvvW7r5q2v5Ukvcm2b5Xs2X+7Gbp36owfi4p4+fyWuUxdNnHz1ULyNuT3DQ835Wv/DC+2Ka7dyf5RJKv3ZDqDs8sfUuSH5h+BfPaqtqxzvFlNWv/l9V3VNU1VfWnVfXNiy7mUEy/br9/knfudWglPrv99C9Zgc8vxs/E+LmsVuHv30qPocs4fq5aQN7q/iTJyd39bUkuy5dmetjcrkry77v7fkmel+SPFlvOwauqY5P8YZKf7u5PLrqeI+0A/Vv6z48kxs9ltRJ//1Z5DF3W8XPVAvIst6r+Ypuq2pbkrkk+tiHVHZ4D9q27P9bdn5uevjjJt29QbRthZW9D3t2f7O5PT9tvSHJMVd1jwWXNrKqOydrg98ruft06TZb6sztQ/5b98xsYP42fS2cV/v6t8hi6zOPnqgXkWW5VfUmS86btRyf5i55Wim9yB+zbXmuSHpG19T6r4pIkT5jO5n1wkk90962LLupIqKp/t2cdZ1WdnrW/l8sQOjLV/ZIk7+3u39pHs6X97Gbp3zJ/fnsxfn6J8XNJLPvfv1UeQ5d9/NxUt5o+XL2PW1VX1a8kubK7L8nah/WKqroxawv/H7u4imc3Y9/+a1U9IsnurPXtiQsr+CBV1auydjbrPapqV5JnJjkmSbr7hUnekLUzeW9M8pkkT1pMpQdvhr49OslPVNXuJP+a5LFLEjqS5CFJHp/kuqq6etr3C0l2Jsv/2WW2/i3z5/dFxk/j52a04uNnstpj6FKPn241DQAAg1VbYgEAAIdFQAYAgIGADAAAAwEZAAAGAjIAAAxW6jJvcDCq6llJPp3ka5K8rbv/fB/tHpnkA919w8ZVB7C5GUNZZWaQ2fK6+5f3NbBPHpnkvhtUDsBSMYayigRktpSq+sWq+kBVvT3JN037XlZVj562n11VN1TVtVX1nKo6I2t31frNqrq6qu5dVT9WVVdU1TVV9YdV9W+G13luVV1eVX+35zWnY/+9qq6bvufZ0757V9Ubq+rdVfVXVXWfDf+BABwEYyhbhSUWbBlV9e1Zu/PXqVn7s39VkncPx782yaOS3Ke7u6qO7+5/rqpLklza3a+d2v1zd//etP1rSZ6c5HnTy5yU5KFJ7pO124O+tqq+L8k5SR7U3Z+pqrtPbS9M8uPd/cGqelCS303y3fP7CQAcOmMoW4mAzFbynUle392fSZJp0B59Islnk7ykqi5Ncuk+XudbpkH9+CTHZu32tXv8UXffkeSGqjpx2vewJL+/5327++NVdWySM5L8wXQb+iS58+F0DmDOjKFsGQIyTLp7d1WdnuSsrN0f/qlZfzbiZUke2d3XVNUTk5w5HPvcsF3Zt6OS/HN3n3oYJQNsGsZQVok1yGwlb0vyyKr6qqo6Lsn3jwenGYm7dvcbkjw9yf2mQ59KctzQ9Lgkt1bVMUkeN8P7XpbkScM6u7t39yeTfKiqfnDaV1V1v/29CMCCGUPZMgRktozuvirJxUmuSfKnSa7Yq8lxSS6tqmuTvD3Jz0z7X53k56rqb6vq3kl+Kck7k7wjyftmeN83Zm0t3ZVVdXWSn50OPS7Jk6vqmiTvydoaO4BNyRjKVlLdvegaAABg0zCDDAAAAwEZAAAGAjIAAAwEZAAAGAjIAAAwEJABAGAgIAMAwOD/A/PnqaeyBVO2AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df = run_query(\"\"\"\n",
    "MATCH (c1:Character)-[:INTERACTS]->(c2:Character)\n",
    "RETURN gds.similarity.euclideanDistance(c1.embedding, c2.embedding) AS distance, 'euclidian' as metric\n",
    "UNION\n",
    "MATCH (c1:Character)-[:INTERACTS]->(c2:Character)\n",
    "RETURN gds.similarity.cosine(c1.embedding, c2.embedding) AS distance, 'cosine' as metric\n",
    "\"\"\")\n",
    "\n",
    "sns.displot(data=df, x='distance', col='metric')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c11725d2",
   "metadata": {
    "id": "c11725d2"
   },
   "source": [
    "Most node pairs that are connected with a relationship have a high cosine similarity. Again, this is expected as the FastRP is designed to translate the network topology structure into embedding space. Therefore, we expect the neighboring nodes in the graph to be very similar in the embedding space. We will examine the node pairs connected in the network with a cosine similarity of less than 0.5 as this is a bit more unexpected. First, we have to tag them with Cypher:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "52687f6e",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 49
    },
    "id": "52687f6e",
    "outputId": "bfc9660d-2a7a-43f0-86e5-7e82cc91f32a"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: []\n",
       "Index: []"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "run_query(\"\"\"\n",
    "MATCH p=(c1:Character)-[i:INTERACTS]->(c2:Character)\n",
    "WHERE gds.similarity.cosine(c1.embedding, c2.embedding) < 0.5\n",
    "SET i.show = True\n",
    "\"\"\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8dd6d88",
   "metadata": {
    "id": "c8dd6d88"
   },
   "source": [
    "It seems that pairs of connected nodes with a lower cosine similarity mainly occur when we have connections between various clusters or communities in the network. If you remember the t-SNE visualization, the nodes in the same community are nicely grouped in the embedding space. However, we have a couple of relationships between nodes from different communities. When we have connections between nodes from various communities, their similarity decreases. It also seems that these nodes have a higher degree, meaning they have many links within their community and then a couple of links to other communities. Therefore, they are more similar to neighbors within their community and then less similar to neighbors from other clusters.\n",
    "# Link Feature Combiner\n",
    "We've got the embeddings ready, and we know that pairs of connected nodes are highly likely to have a high cosine similarity in the embedding space. Now, we will evaluate how different link feature combiners affect the output of the link prediction model.\n",
    "## Cosine combiner\n",
    "Interestingly enough, the first combiner we will take a look at is the Cosine similarity combiner.\n",
    "Cosine link feature combiner. Image by the author.The Link Feature combiner takes pairs of node features and combines them into a single link feature, which is then used as training data to the logistic regression model that will predict new links. We have already done the cosine similarity analysis, so we know that node pairs with a high cosine similarity are likely to be connected. Therefore, you might imagine that new predicted links will be between pairs of not yet connected nodes with high cosine similarity, as this is precisely how our training data looks.\n",
    "The Python script we will be using to produce link predictions is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "6b8e50ff",
   "metadata": {
    "id": "6b8e50ff"
   },
   "outputs": [],
   "source": [
    "def generate_links(combiner, predictedRelType):\n",
    "    # Delete all graph models & drop all named graphs\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.graph.list() YIELD graphName\n",
    "    CALL gds.graph.drop(graphName) YIELD graphName as done\n",
    "    RETURN distinct 'dropped named graphs' as result\n",
    "    UNION\n",
    "    CALL gds.beta.model.list() YIELD modelInfo\n",
    "    CALL gds.beta.model.drop(modelInfo.modelName) YIELD modelInfo as done\n",
    "    RETURN distinct 'dropped ML models' as result\n",
    "    UNION \n",
    "    CALL gds.beta.pipeline.list() YIELD pipelineName\n",
    "    CALL gds.beta.pipeline.drop(pipelineName) YIELD pipelineName AS done\n",
    "    RETURN distinct 'dropped pipelines' AS result\n",
    "    \"\"\")\n",
    "    # Define a new LP pipeline\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.create('lp-pipeline')\n",
    "    \"\"\")\n",
    "    # Define feature combiner\n",
    "    run_query(f\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.addFeature('lp-pipeline', '{combiner}', {{\n",
    "      nodeProperties: ['embedding']\n",
    "    }}) YIELD featureSteps;\n",
    "    \"\"\")\n",
    "    # Define train-test split\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.configureSplit(\n",
    "     'lp-pipeline', {  \n",
    "       testFraction: 0.3,\n",
    "       trainFraction: 0.6,\n",
    "       validationFolds: 7})\n",
    "    YIELD splitConfig;\n",
    "    \"\"\")\n",
    "    # Configure LP model params\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.addLogisticRegression(\n",
    "      'lp-pipeline',  \n",
    "        {tolerance: 0.001, maxEpochs: 500})\n",
    "    YIELD parameterSpace;\n",
    "    \"\"\")\n",
    "    # Add multiple models\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.addLogisticRegression(\n",
    "      'lp-pipeline',  \n",
    "        {penalty:0.001, tolerance: 0.01,  maxEpochs: 500})\n",
    "    YIELD parameterSpace;\n",
    "    \"\"\")\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.addLogisticRegression(\n",
    "      'lp-pipeline',  \n",
    "        {penalty:0.01, tolerance: 0.01, maxEpochs: 500})\n",
    "    YIELD parameterSpace;\n",
    "    \"\"\")\n",
    "    # Construct named graph\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.graph.project('lp-graph', \n",
    "      'Character', \n",
    "      {INTERACTS:{orientation:'UNDIRECTED'}},\n",
    "      {nodeProperties:'embedding'});\n",
    "    \"\"\")\n",
    "    # Train the model\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.train('lp-graph', \n",
    "  {pipeline: 'lp-pipeline',\n",
    "   modelName: 'lp-model',\n",
    "   randomSeed: 42,\n",
    "   targetRelationshipType:\"INTERACTS\"})\n",
    "    YIELD modelInfo\n",
    "    RETURN  modelInfo.bestParameters AS winningModel,  modelInfo.metrics.AUCPR.outerTrain AS trainGraphScore,  modelInfo.metrics.AUCPR.test AS testGraphScore;\n",
    "    \"\"\")\n",
    "    # Predict new relationships\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.predict.mutate('lp-graph', \n",
    "      {modelName: 'lp-model',  \n",
    "       mutateRelationshipType: 'INTERACTS_PREDICTED',\n",
    "       topN: 20,\n",
    "       threshold: 0.45})\n",
    "    YIELD relationshipsWritten;\n",
    "    \"\"\")\n",
    "    # Store relationships back to graph\n",
    "    predicted_links = run_query(f\"\"\"\n",
    "    CALL gds.graph.streamRelationshipProperty('lp-graph', \n",
    "      'probability', \n",
    "      ['INTERACTS_PREDICTED'])\n",
    "    YIELD  sourceNodeId, targetNodeId, propertyValue as probability\n",
    "    WHERE sourceNodeId < targetNodeId\n",
    "    MATCH (s),(t)\n",
    "    WHERE id(s)=sourceNodeId AND id(t)=targetNodeId\n",
    "    MERGE (s)-[:{predictedRelType}]-(t)\n",
    "    RETURN avg(gds.similarity.euclideanDistance(s.embedding, t.embedding)) AS euclidian_similarity,\n",
    "           avg(gds.similarity.cosine(s.embedding, t.embedding)) AS cosine_similarity\n",
    "           \n",
    "    \"\"\")\n",
    "    return predicted_links"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "3f20eb05",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81
    },
    "id": "3f20eb05",
    "outputId": "1ae5f4cf-e4c6-42b8-ccfe-124f192802ce"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>euclidian_similarity</th>\n",
       "      <th>cosine_similarity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.057312</td>\n",
       "      <td>0.999389</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   euclidian_similarity  cosine_similarity\n",
       "0              0.057312           0.999389"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generate_links('cosine', 'PREDICTED_COSINE')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "0337e58d",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81
    },
    "id": "0337e58d",
    "outputId": "d92345db-48a1-447d-e8bd-cd20f98cb47a"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>euclidian_similarity</th>\n",
       "      <th>cosine_similarity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.049519</td>\n",
       "      <td>0.999198</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   euclidian_similarity  cosine_similarity\n",
       "0              0.049519           0.999198"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generate_links('l2', 'PREDICTED_L2')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "5bf881f6",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81
    },
    "id": "5bf881f6",
    "outputId": "3985d3b8-41b1-4713-961d-be15860629af"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>euclidian_similarity</th>\n",
       "      <th>cosine_similarity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.236127</td>\n",
       "      <td>0.992384</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   euclidian_similarity  cosine_similarity\n",
       "0              0.236127           0.992384"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generate_links('hadamard', 'PREDICTED_HADAMARD')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f84e030d",
   "metadata": {
    "id": "f84e030d"
   },
   "source": [
    "# Using multiple Link Feature Combiners\n",
    "In the previous example, we have only used a single link feature combiner. In the case of Cosine or L2 link feature combiner, we have effectively only used a single input feature to the logistic regression model. In practice, it makes sense to have multiple input features that best describe your domain. As a demonstration, we will add the Preferential attachment input as the second link feature. Looking at the documentation in Neo4j, the Preferential attachment is defined as multiplying node degrees between a pair of nodes. In practice, the preferential attachment model assumes that nodes with a higher node degree are more likely to form new connections. Unfortunately, we can't automatically add the preferential attachment link feature just yet, but we can add it manually. To add the preferential attachment input feature, we will first calculate the node degree values for all nodes. You sometimes want to normalize the input features with logistic regression models, so I will show you how to scale features directly in the Link Prediction pipeline. Then we just need to add the Hadamard link feature combiner, which multiplies the input matrices, in this case, node degrees. So if I understand the math correctly, the resulting link feature should represent preferential attachment as we effectively multiply node degrees between pairs of nodes.\n",
    "\n",
    "In the Link Prediction pipeline, you can have as many link feature combiners as you wish. The results of all link feature combiners are then concatenated into a single vector that is used as an input to the Link Prediction logistic regression model. We can add the degree calculation and scaling directly into the pipeline and don't have to prepare the degree features beforehand."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "04ca4efb",
   "metadata": {
    "id": "04ca4efb"
   },
   "outputs": [],
   "source": [
    "def generate_pa_links(combiner, predictedRelType, scale=True):\n",
    "    # Delete all graph models & drop all named graphs\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.graph.list() YIELD graphName\n",
    "    CALL gds.graph.drop(graphName) YIELD graphName as done\n",
    "    RETURN distinct 'dropped named graphs' as result\n",
    "    UNION\n",
    "    CALL gds.beta.model.list() YIELD modelInfo\n",
    "    CALL gds.beta.model.drop(modelInfo.modelName) YIELD modelInfo as done\n",
    "    RETURN distinct 'dropped ML models' as result\n",
    "    UNION \n",
    "    CALL gds.beta.pipeline.list() YIELD pipelineName\n",
    "    CALL gds.beta.pipeline.drop(pipelineName) YIELD pipelineName AS done\n",
    "    RETURN distinct 'dropped pipelines' AS result\n",
    "    \"\"\")\n",
    "    # Define a new LP pipeline\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.create('lp-pipeline')\n",
    "    \"\"\")\n",
    "    # Get the degree value of each node\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.addNodeProperty('lp-pipeline', 'degree', {\n",
    "          mutateProperty: 'degree'\n",
    "    })\"\"\")\n",
    "    if scale:\n",
    "        # Scale the degree using minmax scaler\n",
    "        run_query(\"\"\"\n",
    "        CALL gds.beta.pipeline.linkPrediction.addNodeProperty('lp-pipeline', 'alpha.scaleProperties', {\n",
    "              nodeProperties: ['degree'],\n",
    "              mutateProperty: 'scaledDegree',\n",
    "              scaler:'MinMax'\n",
    "        })\n",
    "        \"\"\")\n",
    "        # Define HADAMARD combiner for node degree combiner\n",
    "        run_query(\"\"\"\n",
    "        CALL gds.beta.pipeline.linkPrediction.addFeature('lp-pipeline', 'HADAMARD', {\n",
    "          nodeProperties: ['scaledDegree']\n",
    "        }) YIELD featureSteps;\n",
    "        \"\"\")\n",
    "    else:\n",
    "        run_query(\"\"\"\n",
    "        CALL gds.beta.pipeline.linkPrediction.addFeature('lp-pipeline', 'HADAMARD', {\n",
    "          nodeProperties: ['degree']\n",
    "        }) YIELD featureSteps;\"\"\")\n",
    "        \n",
    "    # Define feature combiner\n",
    "    run_query(f\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.addFeature('lp-pipeline', '{combiner}', {{\n",
    "      nodeProperties: ['embedding']\n",
    "    }}) YIELD featureSteps;\n",
    "    \"\"\")\n",
    "    # Define train-test split\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.configureSplit(\n",
    "     'lp-pipeline', {  \n",
    "       testFraction: 0.3,\n",
    "       trainFraction: 0.6,\n",
    "       validationFolds: 7})\n",
    "    YIELD splitConfig;\n",
    "    \"\"\")\n",
    "    # Configure LP model params\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.addLogisticRegression(\n",
    "      'lp-pipeline',  \n",
    "        {tolerance: 0.001, maxEpochs: 500})\n",
    "    YIELD parameterSpace;\n",
    "    \"\"\")\n",
    "    # Add multiple models\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.addLogisticRegression(\n",
    "      'lp-pipeline',  \n",
    "        {penalty:0.001, tolerance: 0.01,  maxEpochs: 500})\n",
    "    YIELD parameterSpace;\n",
    "    \"\"\")\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.addLogisticRegression(\n",
    "      'lp-pipeline',  \n",
    "        {penalty:0.01, tolerance: 0.01, maxEpochs: 500})\n",
    "    YIELD parameterSpace;\n",
    "    \"\"\")\n",
    "    # Construct named graph\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.graph.project('lp-graph', \n",
    "      'Character', \n",
    "      {INTERACTS:{orientation:'UNDIRECTED'}},\n",
    "      {nodeProperties:'embedding'});\n",
    "    \"\"\")\n",
    "    # Train the model\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.train('lp-graph', \n",
    "  {pipeline: 'lp-pipeline',\n",
    "   modelName: 'lp-model',\n",
    "   randomSeed: 42,\n",
    "   targetRelationshipType:\"INTERACTS\"})\n",
    "    YIELD modelInfo\n",
    "    RETURN  modelInfo.bestParameters AS winningModel,  modelInfo.metrics.AUCPR.outerTrain AS trainGraphScore,  modelInfo.metrics.AUCPR.test AS testGraphScore;\n",
    "    \"\"\")\n",
    "    # Predict new relationships\n",
    "    run_query(\"\"\"\n",
    "    CALL gds.beta.pipeline.linkPrediction.predict.mutate('lp-graph', \n",
    "      {modelName: 'lp-model',  \n",
    "       mutateRelationshipType: 'INTERACTS_PREDICTED',\n",
    "       topN: 20,\n",
    "       threshold: 0.45})\n",
    "    YIELD relationshipsWritten;\n",
    "    \"\"\")\n",
    "    # Store relationships back to graph\n",
    "    predicted_links = run_query(f\"\"\"\n",
    "    CALL gds.graph.streamRelationshipProperty('lp-graph', \n",
    "      'probability', \n",
    "      ['INTERACTS_PREDICTED'])\n",
    "    YIELD  sourceNodeId, targetNodeId, propertyValue as probability\n",
    "    WHERE sourceNodeId < targetNodeId\n",
    "    MATCH (s),(t)\n",
    "    WHERE id(s)=sourceNodeId AND id(t)=targetNodeId\n",
    "    MERGE (s)-[:{predictedRelType}]-(t)\n",
    "    RETURN avg(gds.similarity.euclideanDistance(s.embedding, t.embedding)) AS euclidian_similarity,\n",
    "           avg(gds.similarity.cosine(s.embedding, t.embedding)) AS cosine_similarity\n",
    "    \"\"\")\n",
    "    return predicted_links"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "34184199",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81
    },
    "id": "34184199",
    "outputId": "a4520ae9-2e2a-431c-dfeb-ac5279b8c419"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>euclidian_similarity</th>\n",
       "      <th>cosine_similarity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.523386</td>\n",
       "      <td>0.963539</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   euclidian_similarity  cosine_similarity\n",
       "0              0.523386           0.963539"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generate_pa_links('cosine', 'PA_COSINE')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "a7ab4362",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81
    },
    "id": "a7ab4362",
    "outputId": "c6044e96-86e5-4aa3-c579-404968d9524c"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>euclidian_similarity</th>\n",
       "      <th>cosine_similarity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.049519</td>\n",
       "      <td>0.999198</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   euclidian_similarity  cosine_similarity\n",
       "0              0.049519           0.999198"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generate_pa_links('l2', 'PA_L2')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "5e6f706c",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81
    },
    "id": "5e6f706c",
    "outputId": "9db77911-2399-4fad-c8ed-4c57d33a194f"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>euclidian_similarity</th>\n",
       "      <th>cosine_similarity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2.101943</td>\n",
       "      <td>0.374777</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   euclidian_similarity  cosine_similarity\n",
       "0              2.101943           0.374777"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generate_pa_links('l2', 'PA_L2_NOTSCALED1', False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "8dbeb46e",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 81
    },
    "id": "8dbeb46e",
    "outputId": "80704aaa-a21a-4f98-cf1a-41d5328c717a"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>euclidian_similarity</th>\n",
       "      <th>cosine_similarity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.30314</td>\n",
       "      <td>0.987134</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   euclidian_similarity  cosine_similarity\n",
       "0               0.30314           0.987134"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "generate_pa_links('hadamard', 'PA_HADAMARD')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af353d42",
   "metadata": {
    "id": "af353d42"
   },
   "source": [
    "# Conclusion\n",
    "I have really enjoyed writing this blog post and learned a lot about FastRP embedding algorithm and Link Prediction pipeline along the way. A quick summary would be:\n",
    "* FastRP is more likely to assign high similarity between neighboring nodes with a low degree\n",
    "* On the other hand, the cosine similarity between connected nodes of different communities could be lower than 0.5\n",
    "* Using multiple link feature combiners can help you better describe your domain\n",
    "* Scaling node features influences the result of Link Prediction logistic regression model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3c058910",
   "metadata": {
    "id": "3c058910"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "colab": {
   "include_colab_link": true,
   "name": "Link prediction combiner.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
